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PREFACE 



This report, Study of the Voluntary Public School Choice Program Interim Report, 
discusses the evaluation findings during the first three years of implementation of the 
Voluntary Public School Choice (VPSC) Program, administered by the U.S. Department of 
Education. 

The report is part of an ongoing national evaluation of the VPSC Program being 
conducted by COSMOS Corporation. Robert K. Yin of COSMOS Corporation is the 
project director, and he, along with Pirkko Ahonen and Dawn Kim, authored the present 
document. The evaluation is supported under a task order and contract with the 
Department. Adrienne Hosek of the Department’s Policy and Program Studies Service 
served as the contracting officer’s representative during the preparation of this report. 




EXECUTIVE SUMMARY 



The National Evaluation of the 

Voiuntary Pubiic Schooi Choice (VPSC) Program 

The No Child Left Behind Act of 2001 (NCLB)^ expanded public school choice 
opportunities for students, particularly for those attending schools in need of improvement. 
The new accountability requirements in Title I of the Elementary and Secondary Education 
Act (ESEA) (Title I, Section 1 1 16[b]) require districts to offer public school choice to 
students in Title I schools that are identified for improvement, corrective action, or 
restructuring as a result of not meeting state definitions for adequate yearly progress (AYP) 
(U.S. Department of Education, 2005).^ In addition. Congress created the VPSC Program 
(Title V, Subpart 3, Section 5241) to support the emergence and growth of choice 
initiatives across the country. The purpose of the program is to assist states and local 
school districts in the development of innovative strategies to expand options for students, 
and to encourage transfers of students from low-performing to higher-performing schools. 

The VPSC Program functions independently from the choice provisions in Title I and 
provides funds to a relatively small number of sites across the country. In October 2002, 
the U.S. Department of Education (the Department) awarded five-year grants to 13 
applicants. Awards ranged in size from $3.4 million to $17.8 million for an average award 
of $9.2 million, or approximately $1.8 mi llion per year. The VPSC-funded sites included: 
the state of Arkansas; Albany, N.Y.; Chicago, 111.; the state of Elorida; Hartsdale, N.Y.; 
Hillsborough County, Ela.; Ea Quinta, Calif.; Miami, Ela.; the state of Minnesota; 

Swanzey, N.H.; New Haven, Conn.; Portland, Oreg.; and Rockford, 111. 

This report presents interim findings from the National Evaluation of the VPSC 
Program, covering the first three years of implementation, from the fall of 2002 through the 
summer of 2005. Over a five-year period, the congressionally mandated evaluation is charged 
with assessing the program’s progress in meeting the goals and fulfilling the intent of the VPSC 
Program’s legislation. As directed by Congress, the evaluation addresses three central 
questions: 

1) What are the characteristics of the VPSC Program’s grantees? 

2) How and to what extent does the VPSC Program promote educational 

equity and excellence? 



* The No Child Left Behind Act of 2001 (P.L. 107-1 10) amended the Elementary and Secondary 
Education Act of 1965. 

2 

The law requires that if all schools in a district have not made adequate yearly progress (AYP) for two 
or more consecutive years, the district shall, “to the extent practicable,” establish a cooperative agreement of 
transfer with other districts in the surrounding area (U.S. Department of Education, February 2004, p. 17). 
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3) What academic achievement is associated with the VPSC Program? 

Question one relates to basic descriptive information about the program sites and their 
implementation strategies, including activities related to community outreach and capacity- 
building within participating schools. 

Question two relates to the stated goals of the VPSC legislation, which stipulated four 
priorities in the selection of sites: 

(a) Provide the widest possible choice to students in participating schools, 

(b) Promote transfers of students from low-performing to higher- 

performing schools, 

(c) Include interdistrict partnerships to allow students to transfer to a 

school in another district from their original school, and 

(d) Require sites to use funds to support student transportation services or 

costs (on the assumption that this would more likely enable students 

to attend more distant schools). 

The extent the program met the goal of providing “the widest possible choice to students” 
is measured by the overall student participation and participation rates in the VPSC 
Program’s choice initiatives. When available, the report also presents data on the numbers 
of transfers from low- to higher-performing schools, and evidence of interdistrict 
partnerships and transportation costs. By design, these priorities do not necessarily reflect 
the only ways in which an evaluation could define “educational equity and excellence.” 
However, they link directly to the legislation’s stated priorities for implementation. 
Exploring other definitions and collecting data to address them were beyond the scope of 
the evaluation. 

Question three pertains to Congress’s interest in having the evaluation investigate the 
achievement outcomes associated with the VPSC Program. The current report includes a 
discussion of the efforts to date to collect achievement data. The results of this analysis 
will be presented in the final report. 



Main Findings 

The evaluation is based on a mixture of quantitative and qualitative methods (see 
chapter 2 of this report). Among other features, it draws from multiple data sources, 
including site visits, surveys, program documents, and student achievement records. Data 
collection will continue through the end of the five-year grant cycle, which ends in the 
spring of 2007. The evaluation data suggest the following interim trends regarding the VPSC 
Program’s choice activities, student enrollment, and progress on its priorities. 
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Characteristics of the VPSC Sites 



The 13 VPSC sites are located in various parts of the country. Ten sites are located in 
predominantly urban areas, two in areas that cover both urban and rural regions, and one in 
an entirely rural area. Ten of the locales represent a residential population of over 100,000 
people each. The public school student populations in the communities are mostly diverse 
and poor. Nonwhite students comprise over 60 percent of the student population at seven 
school systems represented by the sites. Similarly, over 60 percent of the students are 
eligible for the Free and Reduced-Price Lunch program^ at seven of the sites. 

The sites vary greatly in the design of their choice initiatives; they differ widely in the 
number of students served, the number of participating public schools, and the capacity to 
accommodate transfers. In addition, they differ by how they define choice zones and 
manage the flow of students among participating schools. Despite this variation, sites have 
pursued some common paths. 

First, although unique, school choice initiatives tended to fall under four major 
categories based on how sites have defined choice arrangements and directed the flow of 
transferring students: 

• Five of the sites designated specific schools to be either sending schools 

or receiving schools but not both. 

• Five sites defined initiatives whereby the same schools could be both 

sending and receiving. 

• One site established a within-school initiative, in which students choose 

from education programs within the same school and do not transfer 

between schools. 

• Two sites have initiatives that have involved a mixture of the first three 

types. 

Second, all sites focused on two core activities throughout the implementation 
process: 1) engaging parents and community members; and 2) building capacity at schools 
to attract and accommodate choice transfers. As part of the parent and community activity, 
sites’ actions went beyond a rich array of outreach, marketing, and communication efforts. 
The sites also engaged parents and community representatives in developing and 
implementing the choice initiatives. 

3 

The Free and Reduced-Price Lunch program is part of the National School Lunch Program administered 
by the U.S. Department of Agriculture. The program provides nutritionally balanced, low-cost or free 
lunches to more than 26 million students each school day. Children from families with incomes at or below 
185 percent of the poverty level are eligible for free and reduced-price meals. For the period July 1, 2004, 
through June 30, 2005, 185 percent of the poverty level was $34,873 for a family of four (USDA, 2004). 




As part of the capacity-enhancing activity, sites started new academic programs or 
subjects, purchased supplies and equipment for schools, and provided professional 
development to teaching staffs. However, none of the sites reported hiring more staff or 
taking other steps simply to expand the number of seats. The sites’ capacity-enhancing 
activities were therefore not necessarily accompanied by an increase in the number of seats 
or classrooms at the receiving schools. 



Educational Equity and Excellence 

The first three years of the VPSC Program has produced considerable participation on 
the part of enrolling students. The sites’ experiences have been as follows: 

Participants. The numbers of eligible students, applicants, and enrollees in the VPSC 
Program all have increased during the first three years of the program. 

• The number of VPSC sites enrolling students each year has steadily 

increased. Out of 13 sites total, the number has increased from five 
the initial year, to ten and 12 in years 2 and 3 respectively. 

• The capacity within sites to accommodate greater numbers of enrollees 

has increased over time, from 1,087 students in 2002-03 to 7,445 
students in 2003-04, and to 16,163 students in 2004-05 (see figure 1). 

Participation rates. Over the VPSC Program’s first three years, there were also 
increases in the rates of participation, measured by three ratios, the number of: 1) enrolling 
to applying students; 2) applying to eligible students; and 3) enrolling to eligible students. 

• The proportion of enrolling students to those who applied increased from 

47.4 percent in 2003-04 to 56.9 percent in 2004-05. 

• Similarly, the proportion of eligible students who applied increased from 

2.1 percent to 3.3 percent during the same period. 

• The enrollment-to-eligible ratio showed the greatest rate of increase, from 

0.5 percent in 2002-03 and 1.0 percent in 2003-04, to 1.9 percent in 
2004-05. This rate is similar to those experienced at the outset of 
other public school choice initiatives, including those implemented 
under the Title I accountability provisions in the NCLB Act of 2001. 



Variety of choices among educational programs. Progress has been substantial on 
the first of the four priorities. The variety of educational programs within the VPSC- 
funded initiatives and among other non- VPSC initiatives at each site has meant that 
students can choose from a large and diverse number of academic programs. 
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Exhibit ES-1 



Eligible, Applying, and Enrolling Students 
During the First Three Years of the VPSC Program 
(Fall 2002 to Summer 2005) 



Enrollment 
School Year 


No. of sites 
enrolling 
students 


Eligible 


Applying 


Enrolling 


No. of sites 
reporting 


No. of 
students 


No. of sites 
reporting 


No. of 
students* 


No. of sites 
reporting 


No. of 
students 


2002-03 


5 


4 


219,690 


5 


1,391 


5 


** 1,087 


2003-04 


10 


10 


** 755,387 


10 


** 15,721 


10 


** 7,445 


2004-05 


12 


11 


862,396 


11 


28,388 


11 


16,163 



*Applicant count is based on the number of applications received by the sites. 

**Estimates of participant numbers differ from earlier reports. Adjustments to the earlier estimates are 
based on new information from the sites, as of August 2005. 



Transfer of students from low-performing schools to higher-performing schools. 
These transfers are likely to have eomprised only a portion of the students enrolled in the 
VPSC initiatives. First, fewer than half of the VPSC sites have created choice 
arrangements with predesignated sending and receiving schools, and of these, only four 
have limited their enrollment to transfers from low- to higher-performing schools. 

Second, the majority of the sites permit a mixture of transfers that are not limited to 
students moving from low- to higher-performing schools. Moreover, the sites permitting 
this mixture have not always distinguished the portion of transfers from low- to higher- 
performing schools. Of those that did, only 5 percent of their students had transferred 
from low- to higher-performing schools. 

Overall, combining these sites with those with predesignated sending and receiving 
schools, the low- to higher-performing transfers only represent 13.8 percent of the total 
transfers. However, the actual portion could be larger or smaller, depending on the nature 
of the transfers at the sites that did not track or document the pattern of their transfers. 

Implementation of interdistrict approaches. Only four of the 13 sites provide 
interdistrict options, allowing students to transfer to schools outside their home district. 

Student transportation services or costs. Relative to enrollment, transportation costs 
did not increase proportionately as might have been expected, as the VPSC initiatives 
permitted many students who were already attending distant schools to select schools 
closer to home. 



xiii 



Academic Achievement 



The final report for the National Evaluation of the VPSC Program will contain results 
of the analysis of student achievement records collected over the entire period of program 
implementation. The analysis will look at both school and student level performance when 
possible, to assess the achievement outcomes associated with the VPSC Program. 

• Individual-Level Student Achievement Data. The evaluation will attempt 

to compare the achievement outcomes of students enrolled in the 
VPSC-funded initiative to a similar group of students who did not 
enroll. The analysis will be limited to sites that have detailed and 
accurate records for individual students enrolled in the program and 
those in the comparison group. In preparation for the final report, the 
VPSC sites are now receiving technical assistance for collecting 
individual-level student achievement data. However, there are several 
issues with data quality and reliability, which may restrict future 
analysis. 

• Systems-Level Student Achievement Data. The final report will also 

measure performance trends for schools participating in the initiative. 

The achievement outcomes of students in these schools will be 
compared to students in a similar group of schools not participating in 
the initiative. 



Useful Choice Practices 

A major motive underlying the creation of the VPSC Program is the hope that the 
sites’ experiences might serve as models to districts across the country wanting to initiate 
or strengthen their own public school choice options and practices. Even at this early stage, 
several potentially useful practices have emerged from the program. Part of the national 
evaluation’s work has been to document these and other workable strategies. These 
strategies include the design of choice initiatives; parent involvement in choice 
arrangements; and capacity-enhancing activities to augment or strengthen schools’ 
educational programs. In this manner, the VPSC Program’s experiences can ultimately 
help to strengthen public school choice initiatives across the country. 
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1. OVERVIEW 



The National Evaluation of the 

Voiuntary Pubiic Schooi Choice (VPSC) Program 

The No Child Left Behind Act of 2001 {NCLBf^ expanded public school choice 
opportunities for students, particularly for those attending schools in need of improvement. 
The new accountability requirements in Title I of the Elementary and Secondary Education 
Act (ESEA) (Title I, Section 1 1 16[b]) require districts to offer public school choice to 
students in Title I schools that are identified for improvement, corrective action, or 
restructuring as a result of not meeting state definitions for adequate yearly progress (AYP) 
(U.S. Department of Education, 2005).^ In addition. Congress created the VPSC Program 
(Title V, Subpart 3, Section 5241) to support the emergence and growth of choice 
initiatives across the country. The purpose of the program is to assist state and local school 
districts in the development of innovative strategies to expand options for students, and to 
encourage transfers of students from low- to higher-performing schools. 

The VPSC Program functions independently from the choice provisions in Title I and 
provides funds to a relatively small number of sites across the country. In October 2002, 
the U.S. Department of Education (the Department) awarded five-year grants to 13 
applicants (see exhibit 1). Awards ranged in size from $3.4 million to $17.8 mi llion for an 
average award of $9.2 million, or approximately $1.8 million per year. The VPSC-funded 
sites included: the state of Arkansas; Albany, N.Y.; Chicago, 111.; the state of Elorida; 
Hartsdale, N.Y.; Hillsborough County, Ela.; Ea Quinta, Calif.; Miami, Ela.; the state of 
Minnesota; Swanzey, N.H.; New Haven, Conn.; Portland, Oreg.; and Rockford, 111. 

This report presents interim findings from the National evaluation of the VPSC 
Program, covering the first three years of implementation, from the fall of 2002 through the 
summer of 2005. Over a five-year period, the congressionally mandated evaluation is charged 
with assessing the program’s progress in meeting the goals and fulfilling the intent of the VPSC 
Program’s legislation. As directed by Congress, the evaluation addresses three central 
questions: 

1) What are the characteristics of the VPSC Program’s grantees? 

2) How and to what extent does the VPSC Program promote educational 

equity and excellence? 



4 

The No Child Left Behind Act of 2001 (P.L. 107-110) amended the Elementary and Secondary 
Education Act of 1965. 

^ The law requires that if all schools in a district have not made adequate yearly progress (AYP) for two 
or more consecutive years, the district shall, “to the extent practicable,” establish a cooperative agreement of 
transfer with other districts in the surrounding area (U.S. Department of Education, February 2004, p. 17). 
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Exhibit 1 



Location of the Voluntary Public School Choice (VPSC) Grantees 



Portland Public 
Schools, Oreg. 

Award: $6,467,122 



Rockford Public School 
District #205, III. 

Funded: $10,095,054 



State of Minnesota 
Department of 
Education 



Chicago Pubiic 
Schoois, Hi. 



Monadnock 
Regional School 
District, N.H. 

Award: $8,396,542 





\ Award: $11,751,052 


\ 1 Award: $10,225,138 | /i 


New Haven Public 
Schools, Conn. 

Award: $9,509,901 


r 




—K? 




-i. 



Greenburgh 
Central School 
District No. 7, N. Y. 

Award: $2,822,155 



Desert Sands 
Unified Schooi 
District, Calif. 

Award: $7,947,309 



Brighter Choice 
Charter Schools, N. Y. 

Award: $3,411,567 



State of Arkansas 
Department of 
Education 

Award: $9,276,886 



State of Florida 




Miami-Dade 


Department of 


\ 


County Public 


Education 


\ 


Schools, Fla. 


Award: $17,817,910 




Award: $11,717,901 



School District of 
Hillsborough County, Fla. \ 

Award: $10,226,759 



Exhibit Reads: One of the 13 VPSC sites is in Portiand, Oreg., and received a five- 
year VPSC award of $6,467,122. 



3) What academic achievement is associated with the VPSC Program? 

Question one relates to basic descriptive information about the program sites and their 
implementation strategies, including activities related to community outreach and capacity- 
building within participating schools. 

Question two relates to the stated goals of the VPSC legislation, which stipulated four 
priorities in the selection of sites: 

(a) Provide the widest possible choice to students in participating schools. 
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(b) Promote transfers of students from low-performing to higher-performing 
schools, 

(c) Include interdistrict partnerships to allow students to transfer to a school 
in another district from their original school, and 

(d) Require sites to use funds to support student transportation services or 
costs (on the assumption that this would more likely enable students to 
attend more distant schools). 

The extent the program met the goal of providing “the widest possible choice to students” 
is measured by the overall student participation and participation rates in the VPSC 
Program’s choice initiatives. When available, the report also presents data on the numbers 
of transfers from low- to higher-performing schools, and evidence of interdistrict 
partnerships and transportation costs. By design, these priorities do not necessarily reflect 
the only ways in which an evaluation could define “educational equity and excellence.” 
However, they link directly to the legislation’s stated priorities for implementation. 
Exploring other definitions and collecting data to address them were beyond the scope of 
the evaluation. 

Question three pertains to Congress’s interest in having the evaluation investigate the 
achievement outcomes associated with the VPSC Program. The current report includes a 
discussion of the efforts to collect achievement data and a plan for analysis. The results of 
this analysis will be presented in the final report. 

The current report analyzes data from multiple sources using a mixture of quantitative 
and qualitative methods. The evaluation team conducted field studies of VPSC Program 
and comparison sites during the 2003-04 and 2004-05 school years, when team members 
visited administrative offices, parent information centers, and two participating schools at 
each site. In addition, the team carried out surveys of schools involved with the VPSC 
initiatives in both years, to corroborate the implementation of the VPSC initiatives at the 
school level. The evaluation also draws from program documents and student achievement 
records from the first three years of implementation. Data collection will continue through 
the end of the five-year grant cycle, which concludes in the fall of 2007. 

Chapter 1 of the report outlines the purpose of the national evaluation, introduces the 
VPSC Program, and reviews previous research on public school choice. 

Chapter 2 presents the methodology of the national evaluation. 

Chapter 3 provides a detailed analysis of the characteristics of the VPSC Program and 
VPSC sites. This chapter categorizes sites by four types of choice arrangements, describes 
the schools participating in the choice initiatives, and documents the important program 
practices. 
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Chapter 4 reports the progress being made by the VPSC Program to provide greater 
opportunities for students to partieipate in publie sehool ehoiee. The ehapter examines 
trends in eligibility, applieations, and enrollment at the sites. In addition, it diseusses the 
degree to whieh ehoiee has provided students greater opportunity to move from low- 
performing to higher-performing sehools. 

Finally, ehapter 5 of the report summarizes the interim findings and their implieations 
for the program, polieymaking, and future researeh. The ehapter also identifies useful 
ehoiee praetiees that may be emerging from the VPSC Program. The evaluation will 
eontinue to monitor these praetiees as possible suggestions for other distriets to emulate in 
designing their own ehoiee initiatives. 



Previous Research 

Publie sehool ehoiee under the VPSC Program is one of several ways of providing 
students with optional sehooling opportunities (e.g., Raviteh and Viteritti, 1997; and Henig 
and Sugarman, 1999). Other opportunities inelude publiely funded vouehers, tax eredits, or 
tax deduetions that ean be used for publie or private sehools; privately funded sehool 
ehoiee; dual enrollment; home- sehooling; and privately operated publie sehools sueh as the 
Edison sehools (e.g.. Fuller and Elmore, 1996; Hill, Pieree, and Guthrie, 1997; Edueation 
Commission of the States, 2001; Heritage Foundation, 2001; Greene, 2002; Moe, 2003; and 
Betts and Eoveless, 2005). 

All of these efforts seek to improve edueational outeomes. These outeomes inelude 
the opportunity to attend better sehools, hopefully leading to improved student learning. 
Additional benefits inelude improved matehes between edueational offerings and students’ 
interests. 

At the same time, ehoiee initiatives have ereated debate beeause of their possible 
assoeiation with other, less valued by-produets. A major eoneern has to do with the 
possible inerease in soeial stratifieation (i.e., the extent to whieh sehools undesirably differ 
aeeording to raeial or ethnie distinetions) of students aeross sehools,^ either by raeial and 
ethnie distinetions or by soeial elass distinetions. Whether and how sueh stratifieation 
oeeurs (e.g.. Fee and Burkam, 2002; and Clotfelter, Eadd, and Vigdor, 2003) or ean be 
avoided in the design of ehoiee initiatives in the first plaee (e.g., Willie, Edwards, and 
Alves, 2002; Seott, 2005; and Gill, 2005) remains a subjeet of debate and eontinued 
inquiry. 

In some eases, these or other eoneerns have resulted in eommunity resistanee to 
ehoiee options (e.g., Moe, 2003). For instanee, suburban homeowners have expressed 
displeasure over the potential threat to their property values posed by shifting sehool 
enrollments (Neehyba, 2003; and Rebaek, 2005). 



^ To give but one example from a much more extensive literature, a recently completed study of magnet 
schools examined but found little income-based stratification associated with the presence of magnet school 
choice (Archbald, 2004). 
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Public school choice initiatives can vary considerably . Nevertheless, with the hope 
of attaining the claimed benefits, school systems across the country have been 
implementing an increasing array of public school choice initiatives. The initiatives follow 
no single model and vary considerably. Whatever the case, the choices are to be among 
public schools (only) and should be distinguished from initiatives offering choices for both 
public and private schools (including schools with religious affiliations). 

Among the choice initiatives limited to public schools, possibly the most common 
variants, found widely across the country, have been magnet programs and charter schools. 
The magnet programs permit schools to offer a standard curriculum but then to specialize 
in certain subjects or use learning themes to attract students (e.g., Christenson et ah, 2003). 
Charter schools are publicly funded but operate independently of school districts and can 
define their own instructional programs (e.g.. Loveless, 2002; Miron and Nelson, 2002; 
Finnigan et ah, 2004; and Ross, 2005).^ 

Though less frequently found, the broadest arrangement among public schools calls 
for open enrollment. In this arrangement, school systems do not initially assign students to 
any school. Rather, students express their preferences during the preceding spring semester 
by applying to any of the public schools in the system. One of the oldest, districtwide open 
enrollment arrangements has been in place in Cambridge, Mass., since 1981 (Peterkin, 

1991; and Willie, Edwards, and Alves, 2002). As another example, students at Community 
District Four of New York City (Teske et ah, 2000) submit applications, listing up to six 
schools they wish to attend. About 60 percent of the students usually receive their first 
choice (Cookson and Shroff, 1997). 

Another arrangement offers transfer options, in which students can apply to enroll at a 
public school other than their assigned school. Districts may restrict the number of 
applications, based on the availability of seats at receiving schools or on other pre- 
stipulated criteria. The restrictions effectively limit the number of choices and students 
able to participate in the arrangement. When transfers occur, they are considered to be 
from a sending school to a receiving school. A common policy objective is to define low- 
performing schools as the desired set of sending schools and higher-performing schools as 
the desired set of receiving schools. 

Additional variation among choice initiatives arises from the extent of regional 
cooperation in an initiative. The choice arrangements may exist within a single district 
alone (intra-district) or between two or more districts (inter-district). In some cases, choice 
options may cover the whole state, such as the initiatives in Minnesota (Cookson and 
Shroff, 1997; and Boyd, Hare, and Nathan, 2002); Wisconsin (Wisconsin Department of 
Public Instruction, 1999); or Colorado (Benigno, 2000). Conversely, the options may 
pertain only to a small set of schools inside a geographic zone within a district. 
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Because of the nature of the VPSC Program, the review does not try to address the vast literatures on 
specific forms of school choice, including school voucher initiatives, magnet school programs, or charter 
schools. 
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Finally, arrangements vary according to their application processes, selection 
procedures, coverage of transportation and tuition costs, and other administrative 
procedures. For instance, districts can offer choice options to students at every grade level 
or only for certain entry grades. As another administrative variation, once exercising a 
choice, a student may be able to continue through the entire feeder set of schools or may 
have to reapply when moving, from elementary to middle or from middle to high school. 
Districts also may offer different options by grade level. For example, many districts have 
zonal limits for elementary and middle schools but districtwide choices for high schools. 

As a final feature, analysts also have given considerable attention to the nature of outreach 
efforts and the amount and type of the information needed by families and students to make 
appropriate educational choices (e.g., Hamilton and Guin, 2005). 

All these and other variations serve as a backdrop to the initiatives implemented under 
the VPSC Program. The program has provided a continuing opportunity for local districts 
to expand or start new choice initiatives through its funding support and technical 
assistance. The lessons learned from the VPSC Program can suggest improved ways for 
districts across the country to implement the expanded Title I choice options in the future. 

Public school choice has drawn increasing policy interest because of its possible 
association with desired educational benefits at two levels. Such benefits may occur 
regardless of the particular variation in choice arrangement. 

At the first level, the academic performance for participating students may improve, in 
part because they have chosen schools associated with high achievement (e.g., Cullen, 
Jacob, and Levitt, 2005). Student performance also may improve because students are able 
to choose schools with programs more closely match personal and career interests (e.g., 
Hastings, Kane, and Staiger, 2005 and 2006). 

Although an increasing number of studies have examined the benefits to participating 
students, no consistent findings have emerged. For instance, small or no gains in student 
achievement have been reported, and changes in other aspects of student performance, such 
as dropout and suspension rates, also have been mixed (e.g., Clotfelter, Ladd, and Vigdor, 
2003; and Cullen, Jacob, and Levitt, 2005). Overall, one review of choice outcomes by a 
panel of experts concluded that: 

“...Existing research paints a mixed and complicated picture. Choice 
could indeed lead to the benefits its supporters expect, or the harm its 
opponents fear. If so, the effects, both positive and negative, are less 
certain and more situation-dependent than advocates on either side 
acknowledge” (National Working Commission, 2004, p. 23). 



6 




At the second level, the presence of choice options in a system may lead to the 
improvement of every school in the system. Presumably, schools will compete to retain 
and attract students if their budgets are linked to enrollment levels (e.g., Goldhaber et ah, 
2005). Thus, much of the interest in school choice, including public school choice, is 
related to a sensitivity to economic market conditions and greater competition among 
schools, leading to improved performance or what Hoxby (2001) calls “school 
productivity” (e.g., Chubb and Moe, 1990; Belfield and Levin, 2002; Moe, 2003; and Betts, 
2005). 

Competition can affect all schools in a system, even if the number of students actually 
enrolling in a choice initiative is low. For instance, nearly 95 percent of Arizona public 
school students remain in district schools, despite a proliferation of charter schools. The 
charters do not replace the district schools, but they push the district schools to compete, to 
win back the charter parents (Center for Education Reform, cited by National Working 
Commission, 2004, p. 27). Hoxby (2002), noting that choice is about school supply, points 
to the fact that the threat of competition matters and can be demonstrated through economic 
models. Thus, even with a small number of transferring students, all schools in a system 
may still try to improve themselves because of the threat that more students would want to 
leave in the future, should such improvement not occur. 

Systemwide effects are nevertheless not easily detected. The most potent situation 
may occur when students’ tuitions and other costs are transferred directly as a result of 
students exercising choice (e.g.. National Working Commission, 2004). The Seattle Public 
Schools has long used a choice arrangement whereby such costs not only transfer but are 
weighted: Students in special education, as well as in other special categories requiring 
more schooling efforts, have larger tuitions associated with their transfer than students not 
in such categories. The weights are deliberately defined so that schools cannot have a 
balanced budget if they do not attract any such students. Under these circumstances, the 
Seattle system reports that schools do respond competitively (COSMOS, 2004). 

Other choice initiatives that started in the 1990s also have reported systemwide 
effects. These include the open enrollment initiative in Cambridge, Mass., (Peterkin, 1991) 
and the response to a voucher (but not public school) initiative in Milwaukee (Gardner, 
2002). In both of these two cases, the claims included increases in student achievement as 
well as gains in market share (the proportion of all K-12 students in a geographic area 
enrolled in the public school system). Similarly, District Four experienced significantly 
rising, annual test scores in reading and math from 1973 to 1996, compared to all other 
districts in New York City, in conjunction with the district’s expansion of its public school 
choice initiative (Teske et ah, 2000). 

Possibly the clearest system response to the threat of competition was reported by 
Greene (2001) in Florida. Starting in 1999-00, students attending schools that received 
two “F” grades in four years were eligible to receive opportunity scholarships to attend 
other schools. Greene’s study showed that schools receiving their first failing grade, facing 
the prospect of student out-transfers, exhibited exceptionally large gains in the proportion 
of their students passing the state assessment, to avoid receiving a second “F” designation. 
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At the same time, system effects may be small (e.g., Figlio and Rouse, 2006) or may 
not always be found (e.g., Nechyba and Heise, 2000). In addition, some of the systemic 
effects may not be desirable. For instance, schools may spend more time and money on 
public relations and the marketing of schools rather than on the needed academic 
programming (Powers and Cookson, 1999). 

Because of their potential importance, the possibility of changes at either of the two 
levels deserves monitoring. The present evaluation is therefore using individual-level data 
to track possible changes among participating students and school-level data to track 
possible systemwide changes at any given VPSC site. The analyses of individual student 
level data are planned for the final report. 
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2. EVALUATION METHODOLOGY 



The national evaluation follows mixed quantitative and qualitative methods, using 
data from a variety of original and arehival sourees. This breadth of information will help 
to address the three central evaluation questions previously presented in chapter 1 . 



Evaluation Design 

The evaluation design is a nested design. At the school-level, the relevant schools at 
each site can be depicted as a “system of schools.” The systems can consist of any 
combination of the following types of schools: 

(a) Schools /rom which students have transferred (sending schools); 

(b) Schools to which students have transferred (receiving schools); 

(c) Schools to which students have transferred and from which 

students have transferred (same schools are both sending and 
receiving schools); and 

(d) Schools eligible to serve as sending schools, receiving schools, or 

both sending and receiving but in which no transfers may have 
occurred (other eligible schools). 

Aggregate (school-level) data will be collected about the trends in academic 
performance of these different types of schools, to permit comparisons. These data also 
will cover the demographic characteristics of the enrolled students. 

At the student-level in the nested design, the relevant students include the following: 

(a) Students applying for transfer and then transferring (enrollees); 

(b) Students applying for transfer but not transferring, either by their 

own decision or because their applications could not be 
honored — e.g., due to a lack of seats (applicants)', and 

(c) Students eligible to apply (eligible s). 

The national evaluation will use student- level data, gathered by the 13 sites, to compare the 
trends among these groups of students. 

Definition of comparison sites. For each of the VPSC sites, the evaluation team has 
selected a comparison site. The comparisons do not include schools that are any part of the 
“system” of schools participating in a VPSC initiative. 
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In addition to being outside of the “system” of schools, the selected comparison sites 
have several preferred characteristics. First, they were selected based on proximity to the 
VPSC site, using the following criteria: 

• For sites that limit choice to a well-defined zone or zones in their district, 

the comparison site is a similar set of other schools outside of the 
zone(s) but in the same district; 

• For sites whose choice initiatives cover the whole district, the comparison 

site is another district in the same state; 

• For sites that cover multiple districts (e.g., a metropolitan area or a rural 

area), the comparison site is a similar kind of multiple-district area in 
the same state; and 

• For statewide sites, the comparison sites are nonparticipating districts in 

o 

the same state that are similar to the participating districts. 



Second, the comparison sites are comparable to the VPSC sites in academic 
performance, demographic characteristics, and enrollment size. In cases in which all three 
criteria could not be met, priority was given to the first two. 

Third, the comparison sites had few available public school choice opportunities 
similar to the VPSC Program at the time of selection. However, the comparison sites are 
not “no treatment” sites. For instance, across the country, virtually every district offers 
magnet and charter schools, and these same choice programs exist at many of the VPSC 
sites. Similarly, eligible districts also must offer choice related to Title I provisions. Given 
these conditions, the general goal was to identify comparison sites that had no choice 
initiatives resembling those at the VPSC sites, as funded by the U.S. Department of 
Education’s VPSC Program. 

Definition of the initiatives being studied. The national evaluation defines the 
initiatives under evaluation as the ones being supported by VPSC funds. The use of VPSC 
funds has not only defined the initiatives of interest but also has helped focus on specific 
facets of the initiatives. For instance, for some of the 13 sites, the bulk of the funds were 
used to support and enhance parent information centers; other sites have invested heavily in 
parent outreach and media campaigns; yet others have provided funds for new educational 
programs at receiving schools to increase their capacity and attractiveness to serve 
transferring students. The use of funds also has usefully directed the national evaluation’s 
attention to more specific activities, while still attending to the overall VPSC choice 
initiatives at the 13 sites. 



g 

The fact that all districts are eligible to participate may create a complication. However, this 
complication was considered less troublesome than selecting sites from an entirely separate state, where the 
choice and other educational policy conditions could be entirely different. 
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At the same time, there are two complications that diminish the evaluation’s ability to 
determine a direct association between the VPSC Program and school or student outcomes. 

Other choice initiatives at the same site. First, the VPSC-funded initiative may not 
be the only public school choice initiative at a site. In fact, nearly all of the sites already 
had a variety of other initiatives ongoing, at the onset of the VPSC initiative. These other 
options included magnet and charter schools. Title I public school choice, and a variety of 
other choice options offered by the district or state. For instance, several of the VPSC sites 
are located in states that offer open enrollment to all students to attend any public school in 
the entire state. Overall, even when sites used VPSC funds exclusively to support the 
VPSC Program’s initiatives, they had other unrelated choice initiatives operating at the 
same time (see exhibit 2). 



Exhibit 2 

Other Public School Choice Options at the VPSC Program Sites 
(Fall 2002 to Summer 2005) 



Other choice options 


Number of sites with other public school 
choice options in addition to the 
VPSC-funded initiative* 


Magnet schools 


12 


Charter schools 


12 


Title I choice 


10 


Interdistrict options 


7 


Other district options 


7 


Other state options 


7 


Total for all 13 sites 


**55 



*Individual VPSC Program sites can appear under more than one type of choice option. 
**On average, each of the 13 sites had four or more public school choice options in 
addition to the VPSC-funded initiative. 

Sources: Analysis of site visit data and Grant Performance Reports. 

Exhibit Reads: Of the 13 sites, 12 sites had magnet schoois in addition to the 
VPSC-funded initiative. 



Expansion of existing programs. Second, the VPSC Program permitted applicants to 
expand existing arrangements and not just establish new ones. The majority of the sites 
have chosen to expand existing initiatives. Separating the VPSC-funded portion of a 
choice initiative from other aspects of the initiative has not been easy or precisely 
delineated. 
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Data Collection 



The relevant data include: multiple site visits to the VPSC sites; site visits to the 
comparison sites, mainly to confirm the absence of a VPSC-like initiative at the site and to 
document the nature of other choice options that may be in place; surveys covering an 
average of 50 schools at each VPSC site; and the collection of archival data about the 
participating students and the performance of schools in the district (or the site) as a whole. 
The archival data are collected for all of the relevant schools, including those at the 
comparison sites. 

For the site visits and school survey, the instruments include: 

(A) Protocol A: Interview with the VPSC Project Director and Staff (a site 

visit instrument, covering VPSC implementation and trends 
associated with VPSC and largely directed at the VPSC project 
director and staff); 

(B) Protocol B: Interview with Other Participating Persons (a site visit 

instrument directed to school principals, teachers, or parents of 
transferring and non-transferring students); 

(C) Protocol C: Interview with the Comparison Site District and Title I 

Staff (a comparison site instrument, mainly for collecting data from 
district offices); and 

(D) School Survey (a school survey instrument — see appendix — directed to 

principals of schools participating in the VPSC initiatives and to 
administrators of participating districts in the two statewide 
initiatives). 

The list of instruments does not include the collection and analysis of quantitative 
archival data, to address the evaluation questions related to student achievement. School- 
level data come from existing databases compiled by the U.S. Department of Education 
(the Department), providing annual data for nearly every school in the country, starting 
with the year 1998-99. These databases may be supplemented by Web site data from state 
departments of education. Student-level data are compiled and provided to the national 
evaluation by the 13 VPSC sites. 

The completed national evaluation will then have covered the entire five-year VPSC 
Program and have drawn from multiple sources of evidence, as follows: 

• Annual (2002-03 through 2006-07): 

— Original VPSC grant applications (summer 2002 only); 

— Grant performance reports from each VPSC Program site; 

— Individual records of participating students and 
a comparison group; and 
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— School-level records of student achievement trends 
at VPSC Program sites and comparison sites. 

• Selected Years (2003-04, 2004-05, and 2006-07): 

— Field studies of VPSC Program sites and comparisons 
— Surveys of schools participating in the VPSC Program’s initiatives. 



Site visits. The site visits covered the VPSC project site as well as one sending school 
and one receiving school (or two participating schools at those sites not designating 
specific sending and receiving schools). The sites chose two schools with a lot of choice 
activity (i.e., many transferring students), so that the site visit team could observe and learn 
about the VPSC Program’s initiative in action at the school level. 

A two-person team conducted the site visits. During the first two rounds, the site 
visits occurred in the second semester of the school year. These visits included interviews 
of administrative staff from the state or district agency, parent information centers, and any 
affiliated organizations. The third round of site visits will be conducted during 2006-07. 

School survey. School surveys gather data from schools participating in the VPSC 
initiatives and mainly corroborate the participation in the VPSC initiatives by specific 
schools. 

During the first three years of the study, the evaluation team collected survey data from 
410 schools in 2003-04 and 659 schools in 2004-05. At most sites, all of the schools that were 
eligible to participate in the choice initiative were included in the survey. Two statewide 
initiatives were the exceptions. At one statewide site, a random sample of approximately 50 
schools in each of two participating districts completed the survey. At the second site, no 
schools were surveyed; instead, the 12 districts with ten or more students participating in the 
choice initiative were asked to complete the survey at the district level.^ 

The school survey takes the form of a closed-ended questionnaire to the schools 
participating in the VPSC initiatives (see appendix). The survey, directed at the schools’ 
principals, collects information about: student demographics; school performance; the 
choice options available to students; the percentages of students taking part in these 
options; the methods by which choice information is shared with parents; and the receipt of 
professional development by staff members related to school choice. 

Performance reports. The data collection also covers the VPSC grantees’ annual 
performance reports submitted to the VPSC Program Office at the Department. Each year, 
the office issues reporting requirements, based in part on the data collection suggestions by 
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At this site, the district is the appropriate respondent because many of the participating students are not 
assigned to specific schools, either prior to or as part of the VPSC initiative. 

Participating schools were those with students who were either eligible for or enrolled in a VPSC- 
funded initiative. 
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the national evaluation team. During the first three years, the evaluation team suggested 
that the sites provide more detailed descriptions of their VPSC initiatives. Recently, the 
VPSC Program Office has agreed to request more detailed expenditure information from 
the sites in 2005-06 and 2006-07, so that the evaluation team can more thoroughly analyze 
these expenditures. 



Analytic Priorities 

The distinctive contribution of the national evaluation will be to conduct a cross-site 
analysis to arrive at findings for the VPSC Program as a whole, rather than assessing the 
accomplishments at any given site. The nature of the cross-site analysis will vary 
according to the scope of the data supplied by the sites. 

Individual student-level analyses. VPSC sites have been submitting records of 
individual student achievement. The records primarily contain achievement scores of 
students prior to enrolling as part of a VPSC-funded initiative. 

The VPSC sites’ student- level data submissions to date suggest that the sites need 
additional assistance in compiling the desired individual student- level records. The VPSC 
Program is currently engaging a technical assistance provider to help sites compile the 
needed records. 

School-level analyses. To examine systems-level performance at every site, the 
evaluation will rely on sources other than the VPSC sites. The major source, both archival 
and available to the public, is a database compiled by the Department (the National 
Longitudinal School-Level State Assessment Score Database) to be supplemented, when 
needed, by school-level scores posted on states’ Web sites. 

To analyze the data, the national evaluation team has developed and fully tested a 
procedure for aggregating trend data across different sites and states (Yin, Schmidt, and 
Besag, 2006). The procedure aggregates the various trends to estimate an “effect size” for 
the VPSC Program as a whole (Cohen, 1988; and Lipsey and Wilson, 1993). The 
aggregation procedure therefore assumes the form of a meta-analysis, as if findings by the 
VPSC sites were findings from separate research studies. 

The final analysis will compare preprogram and concurrent trends. In addition, the 
analysis will compare the differences with those at the comparison group of (non- VPSC) 
schools. The relevant analysis therefore consists of comparing the “differences [in the 
VPSC and non- VPSC groups] between differences [in preprogram and concurrent trends].” 
The unit of analysis is the school, providing the ability to compare the academic 
performance at different grade-level bands (elementary school, middle school, and high 
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school*^) for the VPSC Program. Finally, separate analyses will examine scores in reading 
and in mathematics. 

Qualitative analyses. The general analytic strategy in dealing with qualitative data is 
to examine the extent to which plausible arguments can be made regarding any relationship 
between a VPSC initiative and subsequent educational outcomes, including student 
achievement. 

The analysis will initially focus on within-site conditions. A key technique will be to 
assemble the VPSC Program’s features, prior to actual data collection, into an evaluation 
framework. The framework should attempt to “explain,” in qualitative terms, how various 
actions and events can theoretically produce the desired outputs relating to educational equity 
and excellence (Evaluation Question Two). Qualitative data from the site visits and other 
documentation will then be used to test the framework’s theory. An important part of this 
procedure is to search for and understand the role of possible rival explanations (Yin, 2000). 

For instance, ongoing district or school policies apart from the VPSC initiative may be strongly 
associated with the observed outputs. 

The qualitative analysis will also consider conditions across sites, looking at trends 
associated with all VPSC sites. For instance, findings and lessons from the VPSC Program 
could reveal different types of choice initiatives, with the 13 sites arrayed into groups based 
on the design of their VPSC-funded choice program. The subgroups could help provide 
greater insight into the associated outcomes from choice initiatives and also serve as 
practical examples to be evaluated by other districts in the future. 

Another analytic priority comes from an expressed need, by the Department, for the 
national evaluation to report new information about useful practices for conducting choice 
initiatives. From the national evaluation, such information on choice initiatives would 
serve at least two audiences: districts implementing the choice provisions of Title I and 
districts wanting to start or strengthen their own public school choice initiatives 
independent of the Title I requirements. 



* ^ The definition of the bands followed the definitions used by the National Center for Education 
Statistics (NCES), accommodating the varied conditions confronted across the country: “elementary” 
includes low grades from pre-K to 3 with high grades up to the 8th grade; “middle” includes low grades from 
4 to 7 with high grades from 4 to 9; and “high” includes low grades from 7 to 12 with a high grade of 12 only. 
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3. VPSC PROGRAM CHARACTERISTICS 



In the VPSC Program, the Department awarded grants to three types of organizations: 
1) nine local or regional school districts; 2) three state education agencies; and 3) one 
nonprofit, charter school organization. Two of the three state education agencies 
contracted with other organizations to implement their choice initiatives. For example, one 
site allocated funds primarily to districts throughout the state to support local school choice 
plans, as well as to an area university to assist these districts with the operation of choice 
information centers. 

The 13 VPSC sites are located in various parts of the country. Ten sites are located in 
predominantly urban areas, two in areas that cover both urban and rural regions, and one in 
an entirely rural area (see exhibit 3). Ten of the locales represent a total population of over 
100,000 people each. The public school student populations in the communities are mostly 
diverse and poor. Nonwhite students comprise over 60 percent of the student population at 
seven school systems represented by the sites. Similarly, over 60 percent of the students 
are eligible for the Free and Reduced-Price Lunch program at seven of the sites. 

The VPSC sites vary greatly in the design of their choice initiatives. They differ 
widely in the number of students served, the number of participating public schools, and 
the capacity to accommodate transfers. In addition, they differ by how they define choice 
zones and manage the flow of students among participating schools (see exhibit 4 for brief 
descriptions). 

Despite this variation, sites have pursued some common paths. First, although unique, 
school choice initiatives tended to fall under four major categories based on how sites have 
defined choice arrangements and directed the flow of transferring students. Second, all 
sites focused on two core activities throughout the implementation process: 1) engaging 
parents and community members; and 2) building capacity at schools to attract and 
accommodate choice transfers. 



Types of Choice Arrangements 

The VPSC Program allows grantees to design choice initiatives to meet their own 
needs. Nevertheless, going beyond the unique circumstances at each site, the VPSC 
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The Free and Reduced-Price Lunch program is part of the National School Lunch Program 
administered by the U.S. Department of Agriculture. The program provides nutritionally balanced, low-cost 
or free lunches to more than 26 million students each school day. Children from families with incomes at or 
below 185 percent of the poverty level are eligible for free and reduced-price meals. For the period July 1, 
2004, through June 30, 2005, 185 percent of the poverty level was $34,873 for a family of four (USDA, 
2004). 
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Exhibit 3 



Context of the VPSC Sites: Community Characteristics 



Characteristic 


No. of sites 


Type of community: 




Urban 


10 


Rural 


1 


Mixed 


2 


Population: 




Under 100,000 


3 


100,000 to Imillion 


6 


Over Imillion 


4 


Public school enrollment: 
No. of students enrolled 




Under 25,000 


4 


25,000-100,000 


4 


Over 100,000 


5 


Percent nonwhite 




Under 30 


1 


30-60 


5 


Over 60 


7 


Percent eUgible for the Free And 
Reduced-Price Uunch program 




Under 30 


1 


30-60 


5 


Over 60 


7 



Source: NCES-Common Core of Data. 



Exhibit Reads: Ten of the VPSC sites are iocated in urban areas. 



Program appears to be supporting four types of choice arrangements (see exhibit 5) that 
provide a fuller understanding of the ways that public school choice can work. 

First, five of the VPSC Program’s initiatives designated specific schools to be either 
sending schools or receiving schools but not both. In this first type, students attending 
sending schools are eligible to transfer, and their choices are limited to a select group of 
receiving schools. At these sites, VPSC funds mainly supported the strengthening or 
capacity-enhancement at the receiving schools. 

Under this first arrangement, four of the sites have defined their sending schools as 
“low-performing” according to the NCLB criteria regarding schools identified for 
improvement. However, because the VPSC legislation does not prescribe a standard for 
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Exhibit 4 



Brief Descriptions of the VPSC Program’s Choice Initiatives 
(Fall 2002 to Summer 2005) 



VPSC grantee 


Award 
amount 
($ million) 


Description of initiative 


1. Arkansas 
Department of 
Education 


$9.3 


Has supported three cohorts of students attending an off-campus (residential or community) 
program to receive a rigorous and comprehensive, self-paced education, delivered online and 
aligned with the state’s standards (requires students to take state assessment). The program 
covered K-5 in 2002-03, K-7 in 2003-04, and K-8 starting in 2004-05. 


2. Brighter Choice 
Charter Schools, 

N.Y. 


$3.4 


Opened its first three charter schools in Albany in 2005-06. In preparation, the site continued to 
educate the public on Title 1 choice; support charter school development; and coordinate 
supplemental educational services in two districts. 


3. Chicago PuhUc 
Schools, III. 


$10.2 


Has supported two cohorts of students (K-8) attending neighborhood learning clusters (NLCs) of 
four-to-six schools each. Four clusters started in 2003-04, three started in 2004-05, and two 
started in 2005-06. There is a new school in each cluster, and existing schools develop 
magnet themes; clusters have coordinators; and schools receive VPSC funds. 


4. Desert Sands 
Unified School 
District, Calif. 


$7.9 


Has supported two cohorts of students transferring from low-performing sending schools to higher- 
performing receiving schools (K-12). Funds help augment curricula at receiving schools with 
environmental studies theme to make them more attractive to transferring students. 


5. Florida 

Department of 
Education 


$17.8 


Has supported student transfers beginning in 2004-05. Funds assist a subset of districts to 
expand choice options (could cover K-12); funds are allocated to mentor districts (already 
successful at choice options) and mentee districts (needing to expand options), and 
postsecondary institutions (to start school choice information and assistance centers). 


6. Greenhurgh 
Central School 
District No. 7, N.Y. 


$2.8 


Has supported two cohorts of middle and high school students (7-12) attending new academies of 
choice in middle and high school. Beginning in 2004-05, three elementary schools 
implemented the International Baccalaureate Primary Years Programme (IB) to offer choice to 
all K-6 students. 


7. Hillshorough 
County School 
District, Fla. 


$10.2 


Has supported a districtwide controlled choice initiative (K-12), starting 2004-05, involving seven 
zones or regions and the creation or expansion of 51 “attractor” programs at existing schools to 
maintain or increase student diversity. Plans for the initiative were in place well before VPSC, 
which only partially supports the initiative. 


8. Miami-Dade 
County Public 
Schools, Fla. 


$11.7 


Has targeted two of eight transportation zones in the district. Provides funds to create choice 
programs at under-enrolled schools (K-12). In 2003-2004, the site opened one new 
“commuter” school in Zone 1 . In 2004-05, the site created or enhanced choice programs at a 
total of nine schools in Zone 1 , and four schools in Zone 2. 


9. Minnesota 
Department of 
Education 


$11.8 


Has supported existing program allowing MPS students qualifying for FRPL to transfer to 56 
schools in eight surrounding suburban districts, and suburban or MPS students to attend 12 
schools in MPS (K-12) with VPSC funding Parent Information Centers, some transportation, 
and support services. 


10. Monadnock 
Regional School 
District, N.H. 


$8.4 


Has supported three cohorts of students making interdistrict transfers (9-12), attending new 
programs in their original schools (6-12), transferring to an alternative high school (MC2) and a 
virtual high school, or enrolling in college courses. Has added new choice programs at two 
high schools and two middle schools. Expansion to elementary school is in progress. 


11. New Haven Public 
Schools, Conn. 


$9.5 


Has supported three cohorts of students transferring from low-performing schools to identified 
higher-performing schools including Lighthouse Schools, magnets, charters, and suburban 
public schools. VPSC funds expanded programs at four Lighthouse Schools (K-6) in 2002-03, 
five Lighthouse Schools in 2003-04, and three in 2004-05. 


12. Portland Public 
Schools, Oreg. 


$6.5 


Has supported two cohorts of transfer students through the augmentation of a districtwide, 
previously available choice program (K-12). VPSC funds help support new enrollment and 
transfer policies; selection, lottery, and transition services; and collaborative curricula planning. 


13. Rockford School 
District #205, Rl. 


$10.1 


Has supported three cohorts of transfer students from low-performing schools (K-8) to identified 
receiving schools, as part of an existing districtwide choice plan. VPSC funds provide support 
to receiving schools; parent and transportation services; and three tutoring programs run by 
community or faith-based organizations. 



Sources; Analysis of site visit data and Grant Performance Reports by COSMOS Corp., 2005. 
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Exhibit 5 



Four Types of Choice Arrangements Implemented by VPSC Sites 



Type of choice arrangement 


No. of VPSC 
sites 


No. of 

participating 

schools 


Predesignated sending or receiving schools 


5 


*60 


Same schools are both sending and receiving schools 


5 


502 


Within-school options only 


1 


6 


Mixture of the first three groups 


2 


**6 


Total 


13 


574 



*Represents the schools at only three of the five sites. The fourth site only started its VPSC-funded 
enrollment in 2005-2006, and the fifth site was enrolling students at predesignated off-campus sites 
(residential or community facilities), but had no predesignated sending schools. 

**Represents the schools at only one of the two sites, because the second site started enrollment in 
2004-05 but did not identify its participating schools. 

Sources; Survey of Schools, Grant Performance Reports, and analysis of site visit data. 

Exhibit Reads: Five VPSC sites have 60 predesignated sending or receiving schoois. 



identifying “higher-performing” schools, the program and its sites have defined the 
receiving schools simply as those that are not “low-performing” schools. At one of the four 
sites, initially eligible receiving schools later became identified for improvement during the 
second year of its VPSC initiative. At that point, the site then ruled those schools to be 
ineligible as receiving schools. 

The fifth site in this first type of arrangement had predesignated sending and receiving 
schools but did not attempt to limit either group according to any low- to high-performing 
criteria. All schools in the system could be a sending school, and the receiving “schools” 
were pre-designated off-campus sites. 

Second, five initiatives defined the same schools as both ‘‘sending” and “receiving. ” 
Under a second type of arrangement, five sites permit transfers between all public schools 
either districtwide or within prespecified zones. In the latter case, students can choose only 
among the public schools located within their assigned zone. Whether delineated by zone 
or district, students can attend any school within that area, regardless of a school’s prior 
performance. This type of choice arrangement gives sites little or no ability to direct the 
flow of students from low- to higher-performing schools. 

An additional observation is noteworthy. At least one site deliberately defined its 
geographic zones to include both schools identified for improvement and higher- 
performing schools. Thus, in a large county wide district, “pie-shaped” zones may have 
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suburban-like, higher-performing schools at their perimeter and urban-like, low-performing 
schools at their center. While the site may limit transfers to within-zone choices, the 
initiative nevertheless gives students the opportunity to transfer from schools identified for 
improvement to higher-performing schools (and from an urban to a more suburban 
environment). 

For both of these first two types of arrangements, the majority of the student bodies in 
all three types of schools (sending-only, receiving-only, or both sending and receiving) 
were nonwhite students and were eligible for the Free and Reduced-Price Lunch program 
(see exhibit 6.) However, in comparing the two types of arrangements, those with the non- 
designated (both sending and receiving) schools had lower proportions of low-income and 
minority students than the sending-only schools in the first type of arrangement; in turn, the 
receiving-only schools tended to have the lowest proportion of low-income students and 
Title I schools. 



Exhibit 6 

Characteristics of Schools Participating in the VPSC Program, 2004-05* 



Characteristics of Schools 


Sending 
schools only 
(n=27) 


Receiving 
schools only 
(n=32) 


Schools that are 
both sending and 
receiving 
(n=288) 




Average percent, per VPSC site: 


Race/ethnicity: nonwhite 


70.6 


58.2 


52.7 


Eligible for the Free and Reduced-Price 


68.5 


51.1 


55.4 


Lunch program 










Perce 


nt of schools that 


are Title 1: 


Title I schools 


63.0 


31.3 


57.3 



*Total number of schools within the VPSC sites is approximately 574. Of those, data for 347 
schools were available for this analysis. 

Source; NCES, Common Core of Data, 2005. 

Exhibit Reads: On average, 70.6 percent of students in sending schoois are nonwhite. 



Third, one site established a within-school initiative, in which students choose from 
education programs within the same school. In this third type, all students remain at the 
original schools. At the single VPSC site that implemented this third type of arrangement, 
students at their high school could choose between two different academic programs that 
had been put into place with VPSC funds. Middle school students had a similar choice. At 
the elementary level, students could choose between two forms of assessment when 
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enrolling in a new International Baccalaureate (IB) option: being graded on the basis of 
either completing a project or taking a test/^ 

Fourth, two initiatives have involved a mixture of the first three types. One VPSC 
initiative encompasses multiple school districts in the same state, with each district defining 
its own choice options. Because the initiative involves 26 districts, the choice options 
cover nearly every variety of arrangement. The second site under this last arrangement had 
a mix of choices, including education programs within the same school as well as transfers 
in which schools could be both sending and receiving. During the site’s initial 
implementation, it also had a designated sending school, which was a high school that had 
been low-performing. The high school was then closed the following year as planned. 

Every VPSC site has other, non-VPSC-funded public school choice initiatives 
operating at the site. These other options include magnet programs and charter schools. 
Title I public school choice, and a variety of other choice options offered by the district or 
state. For instance, several of the VPSC sites are located in states that offer open 
enrollment to all students to attend any public school in the entire state. Overall, even 
when sites supported public school choice initiatives using VPSC funds exclusively, they 
had other unrelated choice programs operating at the same time (see earlier exhibit 2, 
chapter 2). 



Core Activities 

Regardless of the arrangement, all of the sites carried out two core activities. 



Engaging, Notifying, and Reaching Parents and Community 
Members 

Parents and community representatives have played an important role in the 
development and implementation of the choice initiatives. From the beginning, VPSC 
sites invested in outreach to parents and communities to ensure that public school choice 
initiatives met local needs. Sites notified parents and community members of their plans, 
giving them the opportunity to express preferences for educational content, student 
selection criteria, and the design of parent information centers. Among these topics, the 
VPSC sites report focusing on engaging parents and community members in selecting the 
type of educational programming that would best attract students to receiving schools (see 
exhibit 7). 



13 

Although this type of choice deviated somewhat from the more traditional public school choice 
options, the VPSC Program granted the site a special waiver to implement this arrangement. 
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Several VPSC-funded sites also used parent and community input as a way to 
generate public approval for their choice initiatives. At one site, partnership agreements 
between the district and several community agencies were instrumental in developing a 
new environmental studies curriculum. By engaging these groups in the planning and 
implementation of choice initiatives, the sites worked to garner support for their VPSC- 
funded initiatives. 



Exhibit 7 

Efforts to Involve Parents and the Community in the 
VPSC Program’s Choice Initiatives 
(Fall 2002 to Summer 2005) 



Type of involvement 


Sites’ activities 


Establishing the 
initiative 


Market research, surveys of parents, community and parent advisors, 
community focus groups and forums 


Planning the 
initiative 


Parent advisory committees, principai and teacher input, parent representation 
on iottery and parent information center committees 


Implementing the 
initiative 


Parent participation in daiiy instruction, community support for curricuium 

and materiais, parent and student surveys, specialists or counseiors, workshops 
on parenting skiils, technicai assistance to schoois on parental involvement 



Sources; Analysis of site visit data and Grant Performance Reports. 



Exhibit Reads: Sites engaged parents and the community in estabiishing the choice 
initiatives through varied site activities, inciuding: market research; surveys; advisory 
teams; and focus groups. 



As the VPSC sites transitioned from the planning stage to implementation, several 
created ways to keep parents and community members actively engaged. Choice information 
centers have sponsored workshops on parenting skills, health and nutrition, computer 
literacy, and other topics in addition to providing information about public school choice. 

VPSC sites invested in a variety of outreach, marketing, and communication efforts 
targeted at students and families eligible to participate in choice options. With public 
awareness growing, though, many sites have begun to scale back intensive marketing 
campaigns more recently. During the first three years of implementation, the VPSC sites 
used a wide variety of community outlets, including billboards, movie “trailers,” e-mail 
announcements, and Web advertisements (see exhibit 8). An especially ambitious site 
mailed over 100,000 brochures to parents of public school eligible students. In addition, 
sites have invested in training school staff to assist in the communication process. 
Recognizing the importance of raising public awareness, a few sites have entered into 
formal subcontracts with public relations or marketing consultants to increase their 
effectiveness. 



23 



Exhibit 8 



Parent and Community Outreach Activities 
(Fall 2002 to Summer 2005) 



Notification activity 


No. of sites* 


Activities 


Face-to-face parent and 
community outreach 


12 


Districtwide and schooi fairs and open houses, community 
meetings, door-to-door marketing 


Media campaigns 


12 


Advertisements, TV appearances, handouts, Web sites, 
printed resources 


Direct mailing 


10 


Postcards, brochures, newsietters, catalog choice options, 
Appiications 


Parent information 
centers 


10 


Centralized and neighborhood centers, temporary booths at 
area maii or events 


Staff education 


6 


Centralized and schooi-based training, videos and brochures 
to staff 


Total for all 13 sites 


50 


(On average, each of the 13 sites conducted three or more 
types of notification activities.) 



*Individual VPSC Program sites can appear under more than one type of activity. 

Sources: Analysis of site visit data and Grant Performance Reports, COSMOS Corp., 2005. 



Exhibit Reads: Of the 13 sites engaging in community outreach activities, 12 sites 
conducted face-to-face outreach activities that inciuded districtwide or schooi fairs and 
open houses, community meetings, and door-to-door marketing. 



Many of the sites that initially engaged in large-scale public education campaigns 
reported scaling back these efforts as parents, students, and teachers have become aware of 
their choice options. One site decided to eliminate an intensive advertising campaign 
conducted during the first two years of implementation. Sites also have shifted strategies 
over the past two years, concentrating funds previously devoted to districtwide marketing 
campaigns to increasing awareness among the parents of preschool age children. Other 
sites now use communication funds on efforts to increase the capacity of receiving schools 
to accommodate more transferring students. 

A common outreach mechanism has been the development and staffing of parent 
information centers. By 2004-05, ten of the VPSC sites supported at least one parent 
information center to provide parents with information about their choices and to assist them 
in the student application and enrollment process. At one site, several thousand visitors were 
reported during the main enrollment period. However, most sites reported a few hundred to 
several hundred visitors. All the sites reported a significant drop in visitors to the parent 
information centers after the enrollment period was over. 
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Many of the VPSC information centers offer counseling and other services in addition 
to information about VPSC choice initiatives. Some centers market other choice options 
available to students, or serve as the central enrollment offices for their districts. In 2004- 
05, the VPSC sites continued to conduct workshops and educational activities for parents 
primarily through the parent information centers. These activities included GED and 
computer classes, health education workshops, and classes on supporting students’ 
homework activities. 

The district staff at the sites identified the following as desirable characteristics for 
parent information centers: 

• Locations accessible to parents and families; 

• Sufficient and safe parking; 

• Nearby public transportation; and 

• Operating hours that include days and evenings. 



VPSC sites have made ongoing adjustments to the operation of the parent information 
centers as the choice initiatives have continued. At least two sites added parent information 
centers between the 2003-04 and the 2004-05 school years. Another site with several 
parent information centers within the same district reduced the number of centers based on 
use by the community. Yet another site restructured its parent information center, 
reassigning staff to specialty areas dealing with different aspects of the choice initiatives. 

As the sites refine the role of the parent information centers, further analysis is needed to 
determine whether particular models for parent information centers might influence 
participation in choice initiatives. 

VPSC sites made materials available in a variety of languages to communicate with 
families. Over half of the sites had brochures and applications available in English and 
Spanish, and several sites reported using local Spanish-language media outlets 
(newspapers, radio, and television). One site reported that advertising on a local Spanish- 
language radio station was among its most successful outreach efforts. In addition to 
Spanish, other sites reported printing materials in Chinese, Hmong, Lao, Polish, 

Portuguese, Russian, Somali, and Vietnamese. Some sites partnered with other local 
agencies to assist in fielding phone calls from non-English speakers. 

All VPSC sites had ongoing year-round information outlets such as Web sites, 
newsletters, and advertising. The sites worked to implement their public information 
campaigns well in advance of the application deadlines for their choice initiatives. Eor the 
most part, the push took place over one to two months before the application period. 
However, sites also shared information about VPSC-funded choice initiatives as much as 
one year prior to the application period. 
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In some cases, sites delayed notifying parents of their choice options until the state 
had published the names of schools identified as low-performing, which at best occurs the 
summer before the start of the school year. As a consequence, parents and students at these 
sites were notified of their eligibility only a few weeks before they had to make a decision 
on whether to apply to transfer to another school. Similar problems exist with 
implementing Title I public school choice. During 2004-05, one site obtained a final list of 
schools identified for improvement from the state in December 2004, resulting in students 
not being able to transfer until January 2005.^"^ The site estimates that the delay decreased 
the number of students participating in the choice initiative by as much as 80 percent. 

Nevertheless, participating schools believe that parents are aware of their choice 
options (see exhibit 9). Of all schools surveyed, a large proportion indicated their belief 
that most or all parents and families had a good understanding of choice options. This 
response differed, depending on the type of school. At the average site, 63 percent of 
sending schools reported that all or most of parents and families have a good understanding 
of their choice options, as compared to 74 percent of receiving schools and 69 percent of 
schools that are both sending and receiving. However, the results are solely based on the 
school survey and may not represent the views of parents and families. 

Sending schools reported fewer parent notification activities than either receiving 
schools or non-designated schools (see exhibit 10). Among all the schools, the most 
frequent mode of communication reported was through letters to parents and families. 

High percentages of receiving schools and undesignated schools also reported making 
face-to-face contacts with students and their families, through individual or group 
meetings, enrollment fairs, or open houses. In contrast, sending schools reported little such 
direct contact with parents and families. Overall, receiving schools, as compared to 
sending schools, may have been more motivated to provide choice information to parents in 
order to attract students to their programs. Further research is needed to determine whether 
this lack of effort impacted student participation levels. 



Capacity-Enhancing Activities 

The schools participating in the VPSC initiatives have undergone a variety of capacity - 
enhancing activities, including: starting new academic programs or subjects; purchasing 
supplies and equipment; and providing professional development to staff Capacity- 
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The state had made preliminary lists of schools “identified for improvement” available earlier in the 
year, but the site decided to wait for the final list, not released until December; it has decided to work with the 
preliminary lists in future years. 

However, a district may know early in the year that some schools will have to offer choice the next 
school year because the schools are already in improvement status (and would have to make AYP for two 
years in a row to exit). The Department encourages districts to begin notification for choice early when they 
have schools in this situation. 
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Exhibit 9 



Schools Surveyed on Parents’ and Families’ Understanding 
of Choice Options at VPSC Sites, 2004-05* 



School-reported proportion of parents 
and families having a good 
understanding of their choice options 


Average percent per VPSC site 


Sending 
schools only 


Receiving 
schools only 


Schools that are both 
sending and receiving 


All 


31 


24 


19 


Most, over 50 percent 


31 


50 


51 


Some, 20-50 percent 


29 


34 


28 


Few, less than 20 percent 


32 


0 


20 



*N=514 schools for 12 sites enrolling students in the VPSC-funded initiatives in 2004-05. 
Source: Survey of Schools, 2004—05. 



Exhibit Reads: On average, 31 percent of sending schools (per site) reported that all parents 
and families have a good understanding of their choice options. 



enhancing activities included efforts to both accommodate and attract transferring students at 
receiving schools (see exhibit 10). In addition to improvements to existing schools, sites have 
increased capacity within the system by opening new schools, including charter schools. For 
the most part, the new schools were planned in advance of the VPSC initiative, and they should 
not be considered a product of the initiative itself. One site, however, planned the development 
of several charter schools as a key component of its VPSC design. 

Capacity-enhancement activities included investments in new educational programs, 
including orientation assistance, as well as supplemental services that included tutoring, 
after-school programming, and summer school. VPSC funds have helped to support 
counselors and outside organizations to provide these services, which sometimes work in 
conjunction with Title I school improvement efforts. At some sites, schools modified the 
length of the school day or calendar year and rearranged school schedules to accommodate 
additional educational opportunities implemented as part of the VPSC initiative. 

Existing academic programs, as well as other conditions at receiving schools, have 
been important in encouraging enrollment. These VPSC-funded activities notwithstanding, 
schools responding to the school survey indicated their belief that existing academic 
programs were a major factor in students’ decisions to transfer. Similarly, principals, 
teachers, and parents interviewed during site visits to receiving schools overwhelmingly 
responded that the main reason students had decided to transfer to these schools was their 
preexisting reputations for high performance, not necessarily because of any specific 
capacity-enhancing activities completed in connection with the VPSC Program. 
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Exhibit 10 



Efforts to Notify Parents of Their Choice Options, 2004-05* 





Average percent per VPSC site** 


Response category 


Sending 
schools only 


Receiving 
schools only 


Schools that are 
both sending and 
receiving 


Individual, face-to-face meetings 
with school officials 


25 


51 


69 


Group meetings with school officials 


18 


51 


62 


Enrollment fairs or similar events 


32 


54 


69 


Open houses at receiving schools 


18 


75 


61 


Letter mailed to parents and families 


55 


62 


68 


Letter sent home with students 


14 


64 


61 


Announcements in community 
newspapers and other media 


32 


62 


59 


Contacts made by district’s parent 
information center(s) 


37 


51 


40 



*N=555 schools for 12 sites enrolling students in the VPSC-funded initiatives in 2004-05. 
**Individual program sites can appear under more than one category. Average percent 
calculated based on number of sites engaging in the type of activity. 

Source; Survey of Schools, 2004—05. 

Exhibit Reads: On average, 25 percent of sending schoois (per site) reported having 
individuai, face-to-face meetings between parents and schooi officiais reiated to choice 
options. 



Students also may have been attracted by schools’ physical facilities or perceived safety. 
The VPSC legislation expressly prohibits the use of program funds for constmction, but sites 
could use funds from other sources to make physical improvements. At one VPSC site, the 
main receiving school had a combination of new academic activities supported by VPSC funds 
and a new building and classrooms put into place by the school district. At another site, the 
planned receiving schools are charter schools with a similar combination of features. 

Capacity-enhancing services reach both transferring and existing students at 
receiving schools. The new funds have produced improvements that benefit all students at 
receiving schools, including professional development for teachers, science and computer 
equipment, supplies, and additional school staff. In one case, however, the site targeted 
services directly to transferring students by supporting school-based specialists to work 
with them in addition to implementing a new schoolwide curriculum. 
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Exhibit 11 



Schools Involved in Capacity-Enhancing Activities 
at VPSC Sites* 





Average percent per VPSC site 


Type of Capacity-Enhancement 


Sending 
schools only 


Receiving 
schools only 


Schools that are 
both sending and 
receiving 


Becoming a charter school 


NA 


17 


7 


Starting new magnet programs, 
academies, or small learning 
commnnities 


21 


83 


23 


Starting other new academic 
programs or snbjects 


24 


60 


23 


Making other changes in school 
administration (e.g., 
changing school hours) 


26 


22 


13 



*N=555 schools for 12 sites enrolling students in the VPSC-funded initiatives in 2004-05. 
Individual program sites can appear under more than one type of activity. 

Source: Survey of Schools, 2004—05. 



Exhibit Reads: Among the schoois participating in the VPSC Program’s initiatives, on 
average, 17 percent of the receiving schoois (per site) reported becoming a charter schooi. 



Though sites focused capacity-enhancing activities at receiving schools, school survey 
respondents also reported new educational programs at sending schools (see also exhibit 
11). Frequently, these schools were identified as “low-performing” and initiated new 
educational programs to comply with their Title I school improvement plan. The initiation 
of these programs may in some instances have encouraged students to remain at their 
original schools rather than transferring to receiving schools. 

Additional differences were found among sending schools, receiving schools, and 
undesignated schools in the degree of professional development provided to teachers as 
part of the VPSC-funded initiative. The schools surveyed reported that teachers at the 
sending-only schools were less likely to receive professional development about the 
transfer process than other participating schools (see exhibit 12). While only 24 percent of 
the sending-only schools reported receiving such professional development, more than 33 
percent of the receiving-only schools and 30 percent of undesignated schools benefited 
from training specifically related to the school choice transfer process. 
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Undesignated schools are eligible to receive and send transferring students. 
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Exhibit 12 



Teacher Professional Development* 



Survey response category 


A^ 

Sending schools 
only 


erage percent per 

Receiving schools 
only 


VPSC site 

Schools that are both 
sending and receiving 


Teachers received extra staff or 
professional development in 
relation to the transfer 
process 


24 


33 


30 



*N=428 schools for 1 1 of the 12 sites actually enrolling students in the VPSC-funded initiatives 
in 2004-05. 

Source: Survey of Schools, 2004-05. 



Exhibit Reads: On average, 24 percent of sending schoois (per site) 
reported that their teachers received extra professionai development related 
to the transfer process. 



The capacity-enhancing activities were not necessarily accompanied by an increase 
in the number of seats or classrooms at these schools. None of the sites reported hiring 
more staff or taking other steps simply to expand the number of seats and to accommodate 
higher enrollments at a school. Chapter 4 of this report will show that the number of 
applicants exceeded the number of students who were eventually able to enroll in the VPSC 
choice initiatives. Why all applicants could not be served, and whether a lack of available 
seats at receiving schools constrained enrollment, are topics of continuing investigation in 
the present evaluation. 

Only some of the new education programs may be scientifically based. In their 
capacity-enhancing activities, sites did not examine or rely on scientifically based evidence 
to guide their selection or design of education programs. Some of the new programs have 
included nationally recognized educational programs with a record of improving student 
performance (e.g.. International Baccalaureate and Breakthrough to Literacy). Moreover, 
at least one site had developed a curriculum that subsequently received external recognition 
as an exemplary educational model: After only two years of implementation, the site’s 
environmental studies program was nominated by the state for an environmental protection 
agency award. However, evidence about the scientific basis of sites’ other adoptions was 
unavailable, and collecting new data was beyond the scope of this evaluation. 
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4. EDUCATIONAL EQUITY AND EXCELLENCE 



Over the first three years of implementation, student participation and the rate of 
enrollment have increased across sites, and there are signs that the VPSC Program will 
continue to accommodate even more students in upcoming years. Sites have made less 
progress, however, on specific program priorities. 



Enrollment Trends 

The VPSC sites have expanded considerably during the first three years of 
implementation, to serve an ever-increasing number of students. The exact number of 
participants, however, varies according to the definition of “participation.” The evaluation 
tracks the following three possible definitions to measure participation: 

1) Eligible Students: All students who may participate in a VPSC 

initiative; 

2) Applicants: The set of eligible students who apply to attend a 

public school other than their assigned school through a VPSC 

initiative; and 

3) Enrollees: Those students who successfully apply and enroll at a 

school or program of their choice as a result of a VPSC initiative. 

All of these students can be considered participants in the VPSC Program, depending 
on how broadly participation is defined. The most generous definition would count all 
eligible students as participants.*^ Narrower definitions would limit participants to include 
only applicants, or even further, to include only enrollees. The national evaluation tracks 
and reports all three groups of students. 

The numbers of eligible students, applicants, and enrollees in the VPSC Program 
all have increased during the first three years of the program, but overall enrollment 
rates continue to remain low. Both the number of VPSC sites enrolling students each year 
(5, 10, and 12 sites respectively for the first three years), and the capacity within sites to 
accommodate greater transfers have increased over time. In this sense, the VPSC Program 
continued to scale-up during its first three years. The total number of students enrolled in 
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The logic favoring a count of eligible students as participants, even if not applying to transfer to 
another school, is that these students indeed exercised a choice by deciding to stay at their original school. 
Advocates of school choice sometimes use this argument. The inherent assumption is that students who did 
not transfer knowingly made a choice to stay at their original schools. However, no serious attempts have 
been made to verify if students in fact knew they had a choice. 
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the VPSC Program rose from 1,087 students in 2002-03 to 7,445 students in 2003-04, 
and to 16,163 students in 2004-05 (see exhibit 13). 

Exhibit 13 

Eligible, Applying, and Enrolling Students 
During the First Three Years of the VPSC Program 
(Fall 2002 to Summer 2005) 



Enrollment 
School Year 


No. of 
sites 

enrolling 

students 


Elia 

No. of sites 
reporting 


ible 

No. of 
students 


Apa^ 

No. of sites 
reporting 


ying 

No. of 
students* 


Enrol 

No. of sites 
reporting 


ling 

No. of 
students 


2002-03 


5 


4 


219,690 


5 


1,391 


5 


**1,087 


2003-04 


10 


10 


**755,387 


10 


**15,721 


10 


**7,445 


2004-05 


12 


11 


862,396 


11 


28,388 


11 


16,163 



*Applicant count is based on the number of applications received by the sites. 

**Estimates of participant numbers differ from earlier reports. Adjustments to the earlier estimates are 
based on new information from the sites, as of August 2005. 



Exhibit Reads: Four VPSC sites reported a totai of 21 9,690 eiigibie students during 
2002-03. 



At the same time, this rise in enrollment represents a conservative estimate, because 
the twelfth VPSC site did not report enrollment data, and the thirteenth site (needing more 
time to get new charter schools approved and started) implemented its initiative in 2005- 
06, after data collection for this report ended. 

Despite consistent growth as a whole, the rate of participation varied greatly from one 
VPSC site to the next. As discussed previously, some VPSC sites defined eligibility loosely, 
to encompass a much larger population of students than their initiative could reasonably 
accommodate, while others chose a more narrow definition. The number of eligible students 
at each site ranged from 1,842 to 318,231 students (see exhibit 14). The variability resulted 
not only from the size of the site’s VPSC award, but also from the degree to which the site’s 
VPSC-funded activities were embedded within a larger choice initiative. In the latter case, 
the site defined eligibility in terms of its larger public school choice program, which 
substantially increased the number of eligible students. 



1 8 

The 2003-04 figure was originally reported as 3,981 students in last year’s national evaluation report. 
The new figure represents an updating and adjustment mainly because, at the time of that report, one of the 
sites had reported its eligible and applicant but not its enrollment figures. The site has since provided the 
enrollment figure for 2003-04. 
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Exhibit 14 



Site-by-Site Variations in the Numbers of Students Participating 
in the VPSC Program’s Choice Initiatives, 2004-05 





Number of students 




Percent 




VPSC Program 
sites 


Eligible 


Applying 


Enrolling 


Enrolling 
of eligible 


Enrolling of 
applying 


Applying of 
eligible 


A 


108,845 


3,884 


2,755 


2.5 


70.9 


3.6 


B 


29,531 


1 ,037* 


1,037 


3.5 


100.0 


3.5 


C 


1,842 


1,842 


1,842 


100.0 


100.0 


100.0 


D 


318,231 


198* 


198 


0.1 


100.0 


0.1 


E 


4,849 


741* 


741 


15.3 


100.0 


15.3 


F 


300,000 


4,831 


878 


0.3 


18.2 


1.6 


G 


5,224 


149 


34 


0.7 


22.8 


2.9 


H 


4,073 


167 


48 


1.2 


28.7 


4.1 


I 


10,452 


608 


111 


1.1 


18.4 


5.8 


J 


51,000 


5,007 


4,249 


8.3 


84.9 


9.8 


K 


28,349 


9,924 


4,270 


15.1 


43.0 


35.0 


L 




Site is unabie to estimate VPSC-specific enroilment at this time 




M 




No participating students in 2004- 


-05 




Total number of 
students 


862,396 


28,388 


16,163 








Number of sites 
reporting 


11 


11 


11 









*Site had enrollment in 2004-05 but did not report the number of applicants. The present 
analyses assumed the number was at least equal to the number of enrollees. 

**The site has enrollment in a number of different choice options and is developing databases to 
track all choice students. The site is currently unable to provide data specific to participation in 
the VPSC Program’s choice. 

Exhibit Reads: VPSC site A reported having 108,845 eiigibie students in 2004-05. 



The number of eligible students in the VPSC Program is more than 30 times the 
number of applicants and 50 times the number of enrollees, making eligibility the high end 
estimate of the range of participation in the program. As of 2004-05, 862,396 students were 
eligible to participate in public school choice through the VPSC Program. In some cases, 
individual sites defined a large pool of students as eligible, even though the initiative could only 
accommodate much smaller numbers of enrollees. For instance, at one VPSC site, even though 
only a few schools were able to receive transferring students, the initiative defined all students 
in the entire district as eligible to apply to attend these schools. Similarly, another site 
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developed a statewide initiative to serve no more than 450 students, though in principle all of 
the state’s students in kindergarten through eighth grade were eligible to apply. 

From the pool of eligible students, 28,388 applied to schools of choice in 2004-05, and 
less than 60 percent eventually enrolled at these schools. The total number of applicants in 
2004-05 rose 80.6 percent over the previous year, due to the expansion of preexisting VPSC 
initiatives, as well as the increase in the number of sites operating these initiatives. 

Although four of the 1 1 sites enrolled 100 percent of all their applicants (exhibit 14), the 
other seven sites had to develop selection procedures for handling student applications. Five 
sites used a lottery system to select students from the pool of applicants. Two sites used 
another method to prioritize applicants. In this method, the sites prioritized all applying 
students according to their eligibility for Free and Reduced-Price Lunch Program and in 
reverse order of their state achievement test scores (lowest scoring receive highest priority). 
The sites then selected students in this priority order until all openings were filled. 

On the surface, both selection procedures may present promising opportunities for 
comparing choice and non-choice enrollees. The enrollees either had been designated 
randomly (the lotteries) or through a regression discontinuity design (the priority order 
procedure). Therefore, the evaluation team is continuing to investigate the selection 
procedures at the seven sites and plans to report on this and any analyses conducted of these 
sites in the evaluation’s final report. 

Of particular relevance, thus far, is the finding that lotteries have not usually 
represented true random draws. Priorities built into many of the selection procedures had a 
non-randomizing effect. Furthermore, the lotteries have differed, to suit local needs. Important 
variations include: 1) whether students can only apply to a single school or can list several 
schools as options on their application; and 2) whether the district or individual schools retain 
records of the lottery and enrollees. 

The VPSC Program had 16,163 students enrolled at schools of choice in 2004-05, a 
117 percent increase over the previous year. The estimate includes only students 
transferring for the first time in 2004-05, and does not count students who transferred in 
earlier years and who may continue to be enrolled at schools of their choice. Calculating 
the cumulative total depends in part on how many enrollees from a prior year decided to 
return to their original school or enrolled at yet a different school. Due to such attrition, the 
cumulative enrollment lies somewhere between 16,163 and 24,695 students (the first-time 
enrollees in 2004-05 compared to the total number for all years added together). 

As partial clues, two sites have estimated attrition or “dropout” rates over the full 
three-year period of the VPSC Program (from 2002-03 to 2004-05), reporting rates that 
range between 30 and 50 percent. Several other sites have estimated their repeat 
enrollments from 2003-04 to 2004-05, and report continuation rates from 50 to 95 percent. 
However, it is difficult to determine how to count students that leave schools of choice. 
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Some students may have decided to enroll at new schools through the choice initiative, 
while others returned to their assigned school and may be considered true “drop-outs.” Yet, 
many students may have left the district entirely, not necessarily having anything to do with 
their choice experiences. An accurate estimate for cumulative enrollment would need to 
distinguish among these alternatives while also tracking those students that continue to 
attend schools of choice through the VPSC Program. 

Over the VPSC Program’s first three years, there were increases in the rates of 
participation, measured by three ratios, the number of: 1) enrolling-to-applying 
students; 2) applying-to-eligible students; and 3) enrolling-to-eligible students (see 
exhibit 15). The proportion of applying students who enrolled increased from 47.4 percent 
in 2003-04 to 56.9 percent in 2004-05. Similarly, the proportion of eligible students who 
applied increased from 2.1 percent to 3.3 percent during the same period. However, the 
enrollment-to-eligible ratio showed the greatest rate of increase from 0.5 percent in 2002- 
03 and one percent in 2003-04, to 1.9 percent in 2004-05. 

Exhibit 15 

Participation Rates in the VPSC Program 
(Fall 2002 to Summer 2005) 





Proportion of participation (percent) 


Year of 

implementation 


Enrolling 

to 

applying 


Applying 

to 

eligible 


Enrolling 

to 

eligible 


2002-03 


N.A.* 


N.A.* 


0.5 


2003-04 


47.4 


2.1 


1.0 


2004-05 


56.9 


3.3 


1.9 



*In 2002-03, because sites were anxious to implement quickly, they did 
not provide sufficient information on the number of applicants. 

Exhibit Reads: The proportion of appiying students who enroii in the VPSC 
Program’s choice initiatives increased from 47.4 percent in 2003-04 to 56.9 
percent in 2004-05. 



This last rate is similar to those experienced at the outset of other public school choice 
initiatives, including those implemented under the Title I accountability provisions in the 
No Child Left Behind Act of 2001. For instance, one study estimated that one percent of the 
eligible students transferred in 2002-03 and two percent in 2003-04 under the public 
school choice provisions of NCLB (Center on Education Policy, 2004). Similarly, the U.S. 
Government Accountability Office (2004) estimated that 1 percent of eligible students 
transferred in 2003-04 nationwide. Kim and Sunderman (2004) offer one possible 
explanation for the low rates. They found that many students whose applications were 
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granted decided to stay in the neighborhood school because receiving schools might have 
been only marginally better than the failing schools. In some situations, the receiving 
schools were already on the state list for poor performance, although, not yet officially 
listed as low-performing. 

The VPSC Program’s three-year progress shows signs that the program will 
continue to scale-up in its remaining years (see exhibit 16). First, the number of sites 
enrolling students has increased every year (from five sites in the first year, to ten in the 
second, to 12 in the third, and with the 13th site having initiated public school choice in 
2005-06). Second, the amount of VPSC funds spent during the first three years fell short 
of the annual average available for the five-year period (calculated by dividing the total 
program funds by the five-year period). Finally, the rate of expenditures per enrolled 
student has declined in each of the first three years as start-up costs diminished, allowing 
sites to redirect the remaining funds toward site expansion (e.g., increasing the number of 
schools in the initiative). 



Exhibit 16 

Continuing Scale-Up of the VPSC Program, 

Based on the Program’s Existing and Anticipated Expenditure Rate 



Total VPSC Program (all sites) 


2002-03 


2003-04 


Program year 
2004-05* 


2005-06 


2006-07 


Total expenditures ($m) 


11.5 


24.1 


27.6 


28.2 


28.2 










(Estimated 


Remaining) 


No. of enrolling students** 


1,087 


7,445 


16,163 






Expenditures per student ($000) 


10.6 


3.2 


1.7 







*Total expenditures for 2004-05 contains some projections of expenditures by the sites, because the 
reports were submitted in June 2005 for a fiscal year ending in October 2005. 

** Estimates of participant number differ from earlier reports. Adjustments to the earlier estimates are 
based on new information from the sites, as of August 2004. 

Source: VPSC Program Sites’ Grant Performance Reports, “Cover Sheet (ED 524B)” and “Section B: 
Budget information (OMB No. 1890-007),” July 2005. 

Exhibit Reads: During 2002-03, VPSC sites expended a totai of $1 1 .5 miiiion in grant funds, 
an estimated $10,600 per student. 



Annual shifts in the identity of low-performing schools also may create fresh groups 
of eligible students. Demand for public school choice could thus remain constant, if not 
increase, for the life of the initiative. Hence, sites that have linked choice options to school 
performance may experience an increase in the number of participants in their VPSC 
initiative over the next two years, as these sites continue to scale-up. 
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Progress on Program Priorities 



Though sites have successfully expanded the variety of choices available to students 
over the last three years, they have made less progress in other key priorities, discussed in 
the remainder of this section. Three of the priorities derive from the VPSC Program’s 
authorizing legislation: 

• Providing the widest variety of choices to all students in participating 

schools; 

• Promoting the transfer of students in low-performing schools to higher- 

performing schools; and 

• Developing partnerships seeking to implement interdistrict approaches. 



A fourth priority, discussed elsewhere in the legislation, required that the sites use VPSC 
funds to support student transportation services or costs. This requirement reflected the 
assumption that students would be more willing to travel farther from home and consider 
schools from a larger area than if the students had to pay the cost of transportation 
themselves. 

The VPSC Program has been associated with a wide variety of choices among 
educational programs. Progress has been substantial in the first of the four priorities. 
Except for adding new seats, sites have engaged in considerable “capacity-enhancing 
activities,” especially at receiving schools (see earlier exhibit 11, chapter 3). These 
activities have led to the implementation of new educational programs at those schools. In 
addition, sites already were supporting a wide variety of choices before the introduction of 
their VPSC-funded initiatives (see earlier exhibit 2, chapter 2). Together, the variety of 
educational programs within the VPSC-funded initiatives and among other non- VPSC 
initiatives has meant that students can choose from a large and diverse number of academic 
programs. 

The outreach materials describing these choices at most of the sites reveal a wide 
variety of program options. At one urban site, students can choose to participate in any of 
five different choice initiatives, only one of which is the VPSC-funded choice initiative. 
Each initiative involves schools in different neighborhoods, including schools in the 
surrounding suburban neighborhoods. 

Transfers from low- to higher-performing schools are likely to have comprised only 
a portion of the students enrolled in the VPSC initiatives. Eirst, fewer than half of the 
VPSC sites have created choice arrangements with predesignated sending and receiving 
schools, and of these, only four have limited their enrollment to transfers from low- to 



37 




higher-performing schools. However, only three of the four reported enrollment in 2004- 
05. The fourth site began enrolling students in 2005-06 (see row A, exhibit 17). 

Exhibit 17 

Student Transfers from Low- to Higher-Performing Schools, 2004-05 









Transfers from low- to higher- 
performing schools 


Sites’ choice implementation 


No. of 
sites* 


Total 

enrollment 


No. 


Percent 


A. Only targeted students transferring from 
low- to higher-performing schools 


3 


886 


886 


100% 


B. Supported various enrollments, and tracked 
the students transferring from low- to 
higher-performing schools 


3 


8,717 


444 


5% 


C. Supported various enrollments, but did not 
track students transferring from low- to 
higher-performing schools 


5 


6,560 


sites did not track transfers from 
low- to higher-performing schools 


Total 


11 


16,163 


1,330 

(minimum) 


unknown 



*Of the 13 VPSC sites, 1 1 reported enrollment in 2004-05. One site with enrollment was unable to 
estimate VPSC-specific participants at this time and the remaining site was not yet enrolling students 



Exhibit reads: The choice impiementation at three sites targeted students transferring from iow- 
to higher-performing schoois. The sites had a totai enroiiment of 886 students, aii of whom were 
transfers from iow- to higher-performing schoois. 



Second, the other eight sites permitted a wider variety of transfers or had VPSC 
enrollments that involved no transfers. However, only some of these latter sites tracked the 
transfers from low- to higher-performing schools (see row B, exhibit 17), while others did 
not do such tracking (see row C, exhibit 17). (In one case, a site received a waiver from the 
Department to omit such tracking because all of the site’s enrollees were low-performing 
students, but they were not necessarily transferring from low- to higher-performing 
schools.) As a result, the known transfers from low- to higher-performing schools are 
small, but the actual proportion is likely to be greater. 

The sites (in row B) that did track transfers reported only 5 percent of their students 
transferring from low- to higher-performing schools. Overall, the confirmed transfers 
(from rows A and B) represent only 1,330 of 9,603 or 13.8 percent of the total transfers. 
However, the final portion could be larger or smaller, depending on the nature of the 
transfers at the (row C) sites that did not track or document the pattern of their transfers. 
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One possible explanation for this overall pattern might be illustrated by the nature of 
the VPSC-funded initiative at two sites. Both sites used their VPSC funds together with 
funds from other sources to support broader, districtwide public school choice initiatives. 
By creating a broader approach, the sites extended the coverage of their initiatives to the 
full array of their schools and hence provided a wider variety of choices than a VPSC- 
funded initiative alone might have provided. 

At the same time, these districts had proportionately few low-performing schools, 
districtwide. With students able to choose among any school in the system, the bulk of the 
enrollees were highly likely to come from schools not identified for improvement. As a 
result, these large districtwide initiatives reported low percentages of students transferring 
from low-performing to higher-performing schools. Comprising over 50 percent of the 
total VPSC choice enrollment in 2004-05, these two sites dominated the VPSC Program 
and brought the overall rate for such transfers downward. 

Only four of the 13 sites have used VPSC funds to provide interdistrict options, 
which allow students to transfer to schools outside their home district. Regarding this 
third program priority, most of the VPSC sites have limited their choice initiatives to 
within-district options. Moreover, three of the four sites that have interdistrict options put 
them into place before the VPSC Program started. Another VPSC site that originally 
intended to develop interdistrict arrangements did not do so after finding that parents and 
students had minimal interest in such opportunities. 

The VPSC Program gave priority to interdistrict arrangements under the assumption 
that they provide students in low-performing schools with greater opportunities to attend 
higher-performing schools. However, the VPSC Program’s experiences have shown that 
such options can exist in the absence of interdistrict arrangements. In particular, seven of 
the VPSC-funded sites are large urban districts that have diversely performing schools 
within their own district boundaries. One of these districts deliberately divided the district 
into pie-shaped sub-zones. Each zone tended to have higher-performing schools at its 
periphery and lower-performing schools at its center. The district’s choice arrangement 
promoted transfers within (but not between) zones. 

Relative to enrollment, transportation costs did not increase proportionately as 
might have been expected. As stated previously, the legislation required all sites to 
allocate a portion of VPSC funds to support transportation services or costs, to increase the 
number of choices available to students. While nine sites reported using funds for 
transportation, overall these costs did not necessarily increase. A possible explanation is 
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As an additional comparison, another choice initiative, existing contemporaneously with the VPSC 
Program, has been taking place in the D.C. Public Schools. Although this other initiative emphasizes the use 
of vouchers for students to attend private schools and does not involve public school choice, the D.C. 
initiative is nevertheless districtwide and takes place in a district with many schools reputed to be poorly 
performing. Nevertheless, this initiative reported that only 5.8 percent of the students awarded scholarships in 
the spring of 2004 (for enrollment in 2004-05) came from low-performing schools (see Wolf et ah, 2005). 
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that many students already were attending distant schools, and choice now allowed these 
students to select schools closer to home. Under such circumstances, sites experienced 
minimal or reduced transportation costs. 

For example, one VPSC site identified schools in which enrollment had declined in 
part because some neighborhood children were attending more distant schools due to the 
perception of the poor quality of the neighborhood schools. The site designated these 
under-enrolled schools as “receiving schools” and improved their education programs, to 
encourage children to attend their neighborhood school. As another example, two of the 
VPSC sites had recently emerged from court-ordered school busing, from which students 
had been assigned to more distant schools as part of the original desegregation order. The 
VPSC-funded initiative gave affected students the choice of enrolling at neighborhood 
schools. 

In cases in which sites did not require additional funds for transportation, the sites 
received waivers from the Department that exempted them from the original requirement. 
The lesson from these and other examples in the VPSC Program is that transportation costs 
need not increase and may even decrease with the implementation of public school choice, 
depending upon preexisting enrollment patterns and the design of the choice initiative itself 
(see exhibit 18 ). 



Progress in Collecting Data to Examine 
Student Achievement Trends 

As a major evaluation activity, the VPSC Program’s authorizing legislation calls for 
the national evaluation to determine “the effect of the [choice] programs on the academic 
achievement” of participating students, especially those who transfer from Title I AYP 

schools to higher-performing schools — as well as “on the overall quality of participating 
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schools and districts.” 

Fulfilling this requirement calls for data at two levels: individual student records (an 
individual level) and aggregate grade and school- level performance (a systems level). The 
two levels reflect the dual policy aspirations of the legislation and of public school choice 
initiatives. Such initiatives not only benefit individual students but also improve whole 
systems, because of expected marketplace forces whereby schools either improve their 
performance, to compete for student enrollment, or risk undesirable sanctions — such as 
reconstruction and eventual closure. 

Even with the availability of the data at both levels, a critical caveat remains: The 
design of the VPSC Program does not permit the determination of the actual “effect” of the 
program. Many other conditions influencing student achievement coexist with the VPSC- 



See The No Child Left Behind Act of 2001, P.L. 107-110, Title V, Part B, Sec. 5246(b)(3). 
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Exhibit 18 



Changes in Transportation Costs at VPSC Sites 



Changes in 
transportation costs 


Illustrative conditions in VPSC initiatives 


Costs INCREASE (and are 


• Initiative designates receiving schoois that are farther away 


part of the VPSC 


from students’ homes than the sending schoois; 


Program’s expenditnres) 


• Initiative gives students opportunity to transfer to schoois in 




other districts; or 




• Initiative creates increased interest and capacity to attend non- 




neighborhood schoois. 


Sites may experience 


• Initiative is districtwide, and students traveiing greater 


NO CHANGE in costs 


distances may be offset by those traveiing iesser distances; 
or 




• Initiative involves within-school choices oniy, so students 




remain enrolied at originai site. 


Costs DECREASE (relieving 


• Initiative involves a portion of the district, and students 


the VPSC Program from 


previousiy attending more distant schoois may return to 


having such expenditures) 


schoois in this portion; 




• Initiative permits students previousiy bused under 




desegregation to return to cioser schoois; or 




• Initiative permits students to transfer from a schooi to an 




educationai environment closer to home. 


Effect on transportation costs 


• Initiative has not yet been fuiiy designed or impiemented. 


is yet UNKNOWN 





Source: Site visit databases and Grant Performance Reports, COSMOS Corp., 2005. 



funded initiative. At best, the analysis of student achievement data represents an analysis 

of “preprogram” and then “concurrent” trends. The preprogram trends reflect student 

achievement during a multi-year period prior to the start of the VPSC Program’s initiatives. 

The preprogram trends can later be compared with concurrent trends, which reflect student 

21 

achievement during the multiyear period of the VPSC-funded initiatives. 



Throughout the analysis, observations pertain to changes in scores. Such a focus is not synonymous 
with a focus on changes in actual achievement. For instance, improvements in state assessment scores can 
reflect either improving student achievement or weakened criteria in the scoring of criterion-referenced 
assessments. Many state assessments are based on criterion-referenced tests. Despite this shortcoming, no 
other ways of assessing student achievement are as readily available, for schools and districts across the 
country, as the state assessment tests. 
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Individual-Level Student Achievement Data 



At the individual level, the VPSC sites have collected individual records of students 
enrolled in the choice initiatives, but additional clarifications are needed before analysis 
can occur. Every VPSC Program site is contributing to this effort. The ultimate goal is for 
all of the sites to collect individual-level data covering students enrolled in the VPSC- 
funded initiative as well as a comparable group of students who did not enroll. For 
instance, if the choice initiative was oversubscribed, demographically matched students 
who applied but who were not selected for enrollment at a school of choice would be the 
most comparable group of non-transfers. Alternatively, students eligible to apply but not 
submitting an application, matched for demographic characteristics, also could serve as a 
comparison group. 

At the same time, choice initiatives are complicated, leading to the need for additional 
clarification in the student records. The needed clarifications at this time are as follows. 

Sites’ student records submitted to the national evaluation team do not directly 
coincide with the number of enrollees reported elsewhere by the sites. Every VPSC site 
active in enrolling students in a VPSC-supported initiative by 2004-05 has submitted 
individual student records to the national evaluation team. These submissions indicate that 
sites are making serious efforts to assess the performance of the students in their initiatives. 
As of September 2005, the submitted records did not cover the most recent school year 
(2004-05). However, they did cover one or more of the prior years. Since September 
2005, three of the sites have submitted updates, and others should be forthcoming on a 
regular basis. 

The sites’ submissions employ, not unexpectedly, different electronic and database 
formats, also containing different types of information about each student. Reviewing these 
different formats and preparing them for cross-site analysis has required extensive efforts 
on the part of the national evaluation team. A basic check was used to confirm that the 
headcount of records matched the number of students presumed to be enrolled in the site’s 
choice initiative. 

The headcounts thus far indicate that, for most of the VPSC sites, the total number of 
submitted records does not coincide with the total number of enrollees reported elsewhere 
by the sites (see exhibit 19) — e.g., either in their annual grant performance reports 
submitted to the Department or in their reports at the national evaluation’s site visits. In 
some cases, the discrepancies are small, but in most cases the differences are substantially 
greater than 5 percent. The headcounts also do not appear to include the records of any 
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These other reports have formed the basis for all of the three-year estimates reported earlier. Because 
the grant performance reports in particular are official documents submitted to the Department in relation to a 
site’ s accountability for its grant award, the information in these reports has been assumed to be more 
accurate than any tallies from the individual records submitted to the national evaluation team. 
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Exhibit 19 



Number of Student Records Submitted by the Sites, 
Compared to Enrollment Officially Reported By the Sites 



VPSC 

Program 

site 


Year 


Officially 

reported 

enrollment 


No. of 
records 
sent by 
site 


A 


2002-03 


450 


115 




2003-04 


450 


270 




2004-05 


198 


a 


B 


2002-03 


0 


258 




2003-04 


720 


177 




2004-05 


1,842 


a 


C 


Student transfers 
will not start until 2005-06 


D 


2002-03 


0 


0 




2003-04 


1,076 


b 




2004-05 


878 


b 


E 


2002-03 


501 


a 




2003-04 


191 


a 




2004-05 


741 


a 


F 


2001-02 


0 


3,989 




2002-03 


37 


36 




2003-04 


55 


55 




2004-05 


111 


a 


G 


Site cannot yet 
identify participating students 


H 


2002-03 


0 


0 




2003-04 


0 


b 




2004-05 


4,270 


b 



VPSC 

Program 

site 


Year 


Officially 

reported 

enrollment 


No. of 
records 
sent by 
site 


I 


2002-03 


0 


0 




2003-04 


162 


162 




2004-05 


2,755 


a 


J 


2002-03 


19 


12 




2003-04 


31 


62 




2004-05 


48 


b 


K 


2002-03 


0 


0 




2003-04 


3,844 


3,844 




2004-05 


4,249 


a 


L 


2001-02 


0 


23,198 




2002-03 


0 


67,091 




2003-04 


759 


a 




2004-05 


1,037 


a 


M 


2002-03 


80 


67 




2003-04 


157 


149 




2004-05 


34 


a 



Data are being compiled by the site. 

Datasets received from the sites in September 2005; 
records not yet analyzed. 

Exhibit Reads: VPSC Program Site A 
reported an enroiiment of 450 students in 
2002-03, and submitted 115 records. 
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comparison group of students — e.g., students who might have applied to enroll but who 
remained at their original school and therefore were not enrollees in a VPSC-funded 
initiative. 

The existing records need additional details to be used to analyze student 
achievement scores. Inspection of the individual data items within each record also reveals 
the need for additional details. Especially absent has been the identity of the student’s 
school, along with the year of such matriculation. Because sites have kept track of the 
schools participating in their VPSC initiatives, the identification of the school as part of 
each student’s record would in part confirm the student’s enrollment in the VPSC initiative. 

Though not identifying the school, the records from most sites do include information 
about a student’s grade level. 

Similarly, the information in the students’ records does not include the actual calendar 
year for such data as the student’s grade level or, if present in the record, the state 
achievement scores associated with the record. Having the year of testing is especially 
important in those situations where students may only be tested in alternate years, to 
distinguish scores for the preprogram year (the year just before the student enrolled in a 
VPSC initiative) from scores that might be a year older. 

As a further complication, for many of the initiatives in the VPSC Program, students 
may have started to enroll in the initiative while still in their prekindergarten, kindergarten, 
or first grades. For these students, no preenrollment test scores are available because no 
testing on state achievement tests occurred. 

Until sites have clarified the headcounts and these additional details as part of the 
submitted records, further analysis of the student records seems premature. 

To tighten future reporting, the VPSC Program sites might consider expanding and 
improving their databases for tracking their enrollees. The most desired remedy would be 
for sites to strengthen their own, VPSC-specific databases, using information downloaded 
from district files but not relying entirely on such files. 

The core element in these databases would be an individual record for every student 
enrolled in the VPSC initiative for a given academic year — information not necessarily 
present in existing district records and that the VPSC Program sites therefore need to track 
on their own. The sites would verify such enrollment by confirming — a short time after the 
start of the academic year — that the student was indeed attending an eligible VPSC- 
supported receiving school or a school located in a geographic zone associated with the 
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VPSC initiative. Ideally, sites also would have collected application data from these 
students before they were able to enroll at the VPSC-supported school, and the site would 
have had this prior information as part of the individual student’s record. 

Sites would then augment this individual record further, downloading data from 
existing district files, showing, if possible: a) that the same student had enrolled at a 
different school in the prior academic year; and b) the student’s pertinent demographic and 
academic data for as many years as was available in the district records. Using a reliable 
and accurate ID system, the site would then track the same student’s enrollment at the same 
or different schools in subsequent years, continuing to download the relevant test and 
academic data. 

The ongoing database could then help sites to establish an annual headcount of all 
students enrolled in the VPSC initiative. The headcount would now go beyond tallies of 
the first-year enrollees, calculating continuation and dropout rates as well as trends in 
students’ academic performance. 

All of this record-keeping would pertain to a single annual cohort of VPSC enrollees. 
The site would repeat the entire procedure each year, creating a new batch of individual 
records for every new cohort of students applying to and then becoming enrolled in the 
VPSC initiative. Within the same cohort, over time, the site would calculate changes in 
academic achievement. The sites could then calculate the cumulative headcount and other 
characteristics of all of the VPSC enrollees, across all cohorts, by combining its records 
from all of the cohorts. 

Interpreting the achievement levels or trends over time requires some type of 
comparison group — i.e., students in the same grades that did not enroll in the choice 
initiatives at any of the VPSC sites. Sites could define comparison students in a number of 
ways. 

The grossest comparison would be to district or even statewide averages. A more 
telling comparison would be with other students from the same original schools as those of 
the VPSC-supported enrollees but who themselves chose to stay at the same school and not 
to enroll in the VPSC initiative. An even closer comparison might be with the students at 
the same original school who also had applied to enroll in the VPSC initiative but who 
could not enroll because of oversubscription. 
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To avoid threats to maintaining the anonymity of the students, a site would need to establish “school 
codes,” so that only a central source would vouch for the participation of the school in the VPSC initiative, 
but the identity of the school would remain otherwise unknown. 

24 

The academic data could include students’ grades and not just scores on standardized achievement 
tests. The availability of the grades would help overcome a problem with the test data — that they may not be 
available for the year before enrollment in a VPSC initiative because many school systems only test students 
every other year and because first-grade participants might not have any prior testing record. 
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Whichever the comparison group, the sites would create another group of individual 
student records, in much the same fashion as for those enrolled in the VPSC-funded 
initiative. The comparison group also would consist of cohorts, with each cohort of 
students tracked over time and with their academic and demographic information 
downloaded from the routine district records. 



Systems-Level Student Achievement Data 

At the systems level, the national evaluation is comparing student achievement at 
the VPSC sites with state performance and with performance by matched comparison 
sites. The systems level tracking of student achievement relies on aggregate grade- and 
school-level records, not on individual student records. The data consist of scores on state 
assessment tests, for all grades and academic subjects that have been tested. Although the 
data cover every academic subject tested, the national evaluation has focused only on the 
achievement scores in reading and in mathematics. 

Because VPSC Program implementation started in 2002-03, the preprogram trends 
tentatively cover 1998-99 to 2001-02, with 2001-02 serving as the preprogram year. 

The planned analyses will cover two levels — the VPSC sites’ entire district or set of 
districts, and the sites’ targeted schools, if any (e.g., sending and receiving schools). 
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Later analyses will delineate more precisely the preprogram years for the various VPSC Program sites. 
For instance, although preprogram years were 1998-99 to 2001-02 for the VPSC sites that started their 
student enrollment in 2002-03, the years were 2000-01 to 2002-03 for those starting in 2003-04, and the 
years will be staggered even further with the sites starting in later years. Refining the definition of the 
preprogram years will occur once all sites have started their actual implementation. 
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5. INTERIM CONCLUSIONS 



The current report covers the first three years of implementation, focusing primarily 
on identifying the characteristics of the VPSC Program and the extent to which the program 
has made progress in expanding choice opportunities for students. After three years of 
implementation, there is evidence that the VPSC Program has been contributing to an 
increase in the number of schools and students participating in public school choice 
initiatives. Less progress, however, has been made on the legislative priority of 
encouraging transfers from low- to higher-performing schools and in providing out-of- 
district choice options for students. At this mid-program stage, “conclusions” about the 
program also are tentative. 



Preliminary Findings 



What are the characteristics of the VPSC Program’s grantees? 

The VPSC Program awarded a total of 13 grants in 2002 to three types of 
organizations: nine school districts, three state education agencies, and one nonprofit (charter 
school) organization. The initiatives differed widely from each other in design and scope, 
though all focused on two core activities: 1) reaching out to parents and the community; and 
2) building capacity at schools to attract and accommodate choice transfers. Both activities 
appear more prevalent in schools slotted to receive transferring students than in schools that 
were designated as sending-only. The majority of the students in all of the participating 
schools were nonwhite and eligible for the Free and Reduced-Price Lunch program. 

However, the sending-only schools had the largest of these majorities (71 percent nonwhite 
and 69 percent Free and Reduced-Price Lunch program participants) as well as the highest 
proportion of Title I schools (63 percent). 

Sites also varied by how they defined choice zones and managed the flow of students 
among participating schools. These plans can be categorized into four types of choice 
arrangements: predesignated specific sending and receiving schools; geographic areas 
where students can choose any school, and schools may thus be sending and receiving 
simultaneously; within-school options, in which no transfers occur; and a mixture of the 
first three types. Sites have more or less control over the movement of students between 
schools depending on the type of arrangement they adopt. Because choice zones allow 
students to transfer to any school within a select area, they limit sites’ ability to direct the 
flow of students from low- to higher-performing schools. 

The VPSC-funded initiatives have largely existed independent of the Title I choice 
initiatives at the same sites, except for two sites where the initiatives have been 
synonymous. Overall, even when sites support public school choice initiatives using VPSC 
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funds exclusively, they have other unrelated choice programs operating at the same time. 
These options include magnet programs, charter schools, and a variety of other choice 
options offered by the district or state. 



How and to what extent does the VPSC Program promote 
educational equity and excellence? 

The legislation gives priority to programs that provide the greatest choice to students, 
by allowing students to transfer to a wide variety of schools and increasing the capacity of 
those schools to accommodate new students. At the same time, the program has the 
priority of encouraging transfers of students from low- to higher-performing schools and 
providing opportunities for students to transfer to another school outside of their assigned 
district. In some respects, these goals may be contradictory. Arrangements that permit 
students to select any school within a geographic zone may provide a greater number of 
choices but have fewer percentages of students moving from low- to higher-performing 
schools or attending schools in neighboring districts. In contrast, districts can better 
promote transfers from low-performing to higher-performing schools by limiting choice 
options and designating specific sending and receiving schools. 

There is evidence that the VPSC Program has made progress on the first priority to 
provide the widest variety of choice. Over the first three years of implementation, sites 
have expanded the assortment of choice options in participating schools and offer a large 
and diverse number of academic programs to transferring students. Total enrollment in the 
VPSC Program rose from 1,087 students in 2002-03 and 7,445 students in 2003-04, to 
16,163 students in 2004-05. Similarly, the enrollment-to-eligible rate increased from 0.5 
percent in 2002-03 and 1.0 percent in 2003-04, to 1.9 percent in 2004-05. Despite 
consistent growth as a whole, the rate of participation varied greatly from one VPSC site to 
the next, ranging from less than 1 to 100 percent. This variability is explained in part by how 
sites defined eligibility and whether the VPSC-funded initiative was embedded within a 
much larger choice program. 

Less evidence is available regarding progress on the second program priority — 
increasing the number of students transferring from low- to higher-performing schools. 
Only a small number of sites specifically targeted such transfers. Other sites had a wider 
array of transfers and did not necessarily document the proportion moving from low- to 
higher-performing schools. 

Regarding the third program priority — promoting interdistrict transfers — only four of 
the 13 VPSC sites provided interdistrict options, allowing students to transfer to schools 
outside their home district. One site that intended to develop interdistrict agreements 
abandoned this plan after finding a lack of interest in such opportunities among parents and 
students. 
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The legislation also required grantees to reserve a portion of the VPSC funds to cover 
the cost of transporting students to their new school. The evaluation, however, did not find 
evidence that these costs always increased relative to enrollment, as the VPSC initiatives 
permitted many students, already attending distant schools, to select schools closer to 
home. 



Useful Choice Practices 

A major motive underlying the creation of the VPSC Program was the hope that the 
sites’ experiences might develop useful choice practices to help states and districts across 
the country wanting to initiate or strengthen their own public school choice options and 
procedures. Even at this early stage, several potentially useful practices have emerged 
from the program. 

The breadth of VPSC Program’s experiences will potentially be useful to other districts 
in the future. The program covers a variety of choice arrangements. Although the program 
has only 13 sites, they cover a full range of communities and educational situations (e.g., large 
urban districts compared to small rural districts compared to districts with heterogeneous 
groups of schools compared to statewide initiatives). The breadth of the VPSC Program’s 
experiences is likely to serve the diverse conditions found in districts across the country. 

As one of the most helpful experiences, the program is showing how four different 
choice arrangements can work. The experiences with these varied arrangements, along 
with eventual data about their experiences, will create a solid body of knowledge. Other 
districts can draw from these experiences in the future. 

To make knowledge about the four arrangements as useful as possible, the present 
evaluation’s final report will update the experiences with each type of arrangement, along 
with the VPSC Program’s other experiences, at the end of the 2006-07 school year. 

The examination will cover the enrollment and other participation associated with 
each type of arrangement, as well as how each type addresses the important priorities of 
choice initiatives (providing the widest variety of choices to all students in participating 
schools; promoting the transfer of students in low-performing schools to higher-performing 
schools; developing interdistrict partnerships; and making funds available to support 
transportation services). As a related inquiry, the analysis also will study the possible 
relationships among these priorities, and in particular, any potential complimentarity or 
conflict among them. 

The various sites also are initiating and further refining specific choice-related 
practices potentially useful for other districts. These potentially useful practices cover: 
the design of choice initiatives; parent involvement in choice arrangements; and capacity- 
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enhancing activities to augment or strengthen schools’ educational programs (see exhibit 

20 ). 



Exhibit 20 

Potentially Useful Choice Practices from the VPSC Program 



Designing Choice Initiatives 

Working Outside the System: Creating a “community movement” and mobilizing families to be 
aware of the performance of their schools, leading to the promotion of new choices — e.g., the creation of 
charter schools. 

Maximizing Choice Options Within Small Geographic Areas: Defining subdistrict “zones” to 
include the most heterogeneous group of schools possible, so students can have wide choice but minimize 
transportation costs. 

Empowering Schools to “Own” a Choice Initiative: Developing a formal process for soliciting 
schools’ participation, to encourage them to take greater ownership over a choice initiative. 



Parent Involvement 

Gaining “Input” from Families: Using large-scale surveys, media campaigns, and public forums to 
raise public awareness and participation in the choice initiative. 

Operating Parent Information Centers: Designing and staffing multiple parent information centers 
to serve a large community. 

Designing User-Friendly Communication about Choice Options: Creating easily searchable 
electronic or hardcopy materials, presenting students and their families with different kinds of choice 
options. 



Capacity-Enhancing Activities 

Defining an Array of Educational Programs to Promote Student Diversity: Attending to the 
offerings at different schools to assure that the programs attract diverse student populations. 

Developing Formal Community Partnerships: Forming agreements with over 30 local institutions 
including local universities, businesses, and nonprofit agencies, to provide support for new educational 
programs at schools in the choice initiative. 

Expanding Distant Learning at “Off-Campus” Locations: Developing a K-8 curriculum for off- 
campus, online learning that is aligned with state standards and state assessments. 



As an example of a potentially useful strategy for increasing parent notification, one 
VPSC site has developed a user-friendly, online database. Parents and students can tap this 
database and electronically search for information about their choice options. Because 
many districts have a large variety of choice options, the amount of information available 
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to parents and students may be overwhelming. By organizing this information online, the 
site has made it easier for parents and students to quickly learn about their choice options. 



Rather than providing information electronically, another VPSC site has distributed 
hardcopies of materials to notify parents of the variety of choice options available. Over 
the course of implementation, the site converted its materials from a program- specific 
format (e.g., separate brochures covering magnet schools, charter schools, the VPSC- 
funded initiative, and other choice initiatives at the site) to a school-specific format (e.g., 
separate brochures for each school, describing all of the choice options available at the 
school), after they found that parents usually had an idea about the schools they might 
consider. The latter format has allowed parents to quickly collect information about the 
programs offered at these schools, rather than have to search through materials about all 
programs, some of which were not relevant to their interests. 

As one example of a promising approach to enhance school capacity, another of the 
VPSC sites has developed a broad-based set of partnerships with local businesses, community 
groups, and institutions of higher education (IHE). These partners all have contributed to the 
capacity-enhancement process, helping the district to strengthen new educational programs. 
Moreover, the partnership, developed as a result of the VPSC Program, has successfully 
garnered additional funds to expand educational programs even further. 

Part of the national evaluation’s work will be to document and disseminate these and 
other useful strategies and practices. In this manner, the VPSC Program’s experiences can 
ultimately help to strengthen public school choice initiatives across the country. 
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OMB Clearance #1875-0224 
Expiration Date: 10-31-09 



COVER SHEET 

QUESTIONNAIRE FOR INDIVIDUAL SCHOOLS 
National Evaluation of the Voluntary Public School Choice Program 



Date: 
Respondent: 
Title: 
Phone: 
School Name: 
District: 
State: 



FOR NATIONAL EVALUATION USE 



ID Number: 



Date Received: 



School Survey, 2006-07 



COSMOS Corporation 
Phone: 301-215-9100 
Fax: 301-215-6969 









OMB Clearance #1875-0224 
Expiration Date: 10-31-09 

QUESTIONNAIRE FOR INDIVIDUAL SCHOOLS 

1. School’s Name, Address, and Grade Levels: 

Name: 

Address: 

{circle lowest and highest): pre-K K1234567891011 12 

2. The following data were calculated based on student enrollment from which school semester? 

{check ONE only): 

for Spring 2006 

for Fall 2006 

for Spring 2007 

Other date (specify the date) 

Total No. of Students: {no.) 



American Indian or Alaska Native: 


(%) 


Asian: 


(%) 


Black or African American: 


(%) 


Hispanic: 


(%) 


Native Hawaiian or other Pac. Isid.: 


(%) 


White: 


(%) 


Other: 


(%) 


Eligible for Free and Reduced Price Lunch: 


(%) 


With lEP: 


(%) 


W iili Limited English Proficiency: 


(%) 


Migrant: 


(%) 



3. Does this School receive Title I assistance? Yes (schoolwide) Yes (targeted) N o 

If yes, has the school been identified as failing to make Adequate Yearly Progress, based on 
student achievement scores for the: 

for the 2006-2007 school year 

for the 2005-2006 school year 

for the 2004-2005 school year 

none of the above 

2 
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4. School Choice Options for Students at this School; 

{check ALL that apply): 

a. students within the district may transfer to this school 

b. students outside the district may transfer to this school 

c. students may transfer from this school to other schools within the district 

d. students may transfer from this school to other schools outside the district 

e. Other - e.g., open enrollment (please explain): 

f. there are no school choice options (If you answered “f”, please stop; do not 
respond to the remaining survey items.) 

(if you checked items a, b, c, or d): 

For 2006-07, about how many students have transferred ? 

g. (number of students transferring to this school) 

h. (number of students transferring from this school) 

i. Did teachers receive any extra staff or professional development, 

in relation to the transfer process? Yes N o 

]■ If yes, what were the main topics of the staff or professional development? 

(topic 1) 

(topic 2) 

5. Notifying Parents about School Choice Options: 

What actions did the school take to notify parents/families of their choice options? 

(check ALL that apply) 

a. individual, face-to-face meetings with school officials 

b. group meetings with school officials 

c. enrollment fairs or similar events for parents to learn about choice options 

d. open houses at receiving schools 

e. letter mailed to parents/families 

f. letter sent home with students 

g. announcements in community newspapers or other media 

h. contacts made by the district’s parent information center(s) 

i. other (please explain): 

j How many languages, other than English, have been used in these notification procedures? 
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6. In your opinion, what proportion of the parents/families had a good understanding of their 
choice options last year? {check ONE only) 

a. all parents/families 

h. most parents/families (e.g., over 50 percent) 

c. some parents/families (e.g., between 20-50 percent) 

d. few parents/families (e.g., less than 20 percent) 

If you checked 6h, 6c, or 6d, what is the most important thing you can 
recommend, to 

improve parents/families’ understanding of their choice options? 



7. Has your school started new programs (e.g., magnets, academies, small learning communities, 
new academic subjects) to be more attractive, either to reduce the number of students transferring 
out or to increase the number transferring in? {check ALL that apply) 

a. becoming a charter school 

b. starting new magnets, academies, or small learning communities 

c. starting other new academic programs or subjects 

d. making other changes in school administration (e.g., changing school 

e. hours) 

other 

f. no new programs 

Briefly describe the new programs and the main changes in school operation and 
administration. 
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